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ABSTRACT 


This thesis examines the potential effect of unmanned aerial vehicles (UAV) in 
the Decisive Stonn operations. This analysis of UAV technology will assist Gulf 
Coalition Forces decision makers in their selection of the most suitable and cost-effective 
unmanned aerial vehicles to support detection operations. We use Map Aware Non- 
Uniform Automata, an agent-based simulation software platform, for the computational 
experiments. It models the operational area, system entities for the Gulf Coalition 
Forces, and the Houthi militia’s attempts to cross the Saudi-Yemeni border. The 
software collects relevant data that can be translated into measures of mission 
effectiveness. Results from 10,400 simulation runs of Houthi efforts to infiltrate the 
operational area are analyzed using descriptive statistics, linear regression, and 
partition trees. These results, which include a significantly increased percentage of 
infiltrators being detected by Gulf Coalition Forces and improved time to detect 
them, support the use of UAVs in detection missions. Computer experimentations 
and analyses reveal the most significant capabilities the UAV should have 
to achieve operational goals. These significant capabilities fall within the U.S. 
Department of Defense’s Group 3 UAV classification. Therefore, the Gulf Coalition 
Forces should consider procuring UAVs in that category to enhance detection in the 
operational area. 
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EXECUTIVE SUMMARY 


Saudi Arabia leads the Gulf Coalition Force (Bahrain, United Arab Emirates, 
Qatar, and Kuwait) in executing Operation Decisive Storm, a military operation in 
Yemen against Houthi militias and their allies (Adaki 2015; Mello and Knights 2016). 
This operation defends the Yemeni government and protects the borders of northern 
Yemen’s and the Kingdom of Saudi Arabia (Al Jazeera 2016). 

Detecting the militia’s scouts, infiltrators, and terrorists before they perform their 
reconnaissance or attack is critical for defending the border and Gulf Coalition personnel. 
David Roberts (2012) notes that during the Gulf Coalition Force (GCF) operation in 
Yemen, the Houthi militias successfully gathered location information about the Emirati 
and Bahraini troops. With this information, the Houthi militias were able to fire a Soviet 
Tochka missile that killed more than 50 soldiers. Other attacks have also resulted in 
losses to the GCF, frequently because of the force’s inability to detect those infiltrators. 

This research examines the potential effects of employing unmanned aerial 
vehicles (UAV) in the Decisive Stonn Operations in Yemen. Results will assist decision 
makers in their acquisition selection of an appropriate UAV to support detection 
operations. Furthennore, this study shows how the GCF participating in Yemeni 
operations can improve border security and defend their operational areas against enemy 
incursions and reconnaissance. This thesis analyzes the use of the UAVs in conjunction 
with detection methods already being employed by ground forces for the GCF’s Decisive 
Storm operational area. Using the UAVs in the Decisive Stonn operational area might 
increase the probability of detection of those infiltrators and militia members. 

This thesis uses Map Aware Non-Uniform Automata (MANA), an agent-based 
simulation software platform. First, we examine the effect of introducing UAV 
technology into the detection operations. Applying computer experimentation to model 
the Houthi militia’s attempts to cross the border provides insights into the Gulf Coalition 
Forces’ intelligence, surveillance, and reconnaissance (ISR) capabilities. Additional 
experimental designs effectively explore the required UAV characteristics that can 
enhance the detection mission. Regression analysis and partition tree analysis provide 

xvii 



additional assistance in examining all design point options. Finally, a relative cost and 
benefit analysis helps in identifying the design option we wish to pursue. 

The introduction of unmanned aerial vehicles supporting the detection missions in 
the Decisive Storm operation reveals significant improvements in the percentage of 
infiltrators that the Gulf Coalitions Forces detect as well as in the time it takes to detect 
them. Nevertheless, in the first introduction, the UAVs lacked the necessary capabilities 
and thus were unable to support GCF detection operations. Therefore, we analyzed more 
computer experimentation to discover the UAV characteristics necessary to satisfy the 
GCF operational goals. Using regression analysis we sought to uncover the most 
significant factors having an impact on the UAV’s detection capabilities; this analysis 
indicated six such factors. Furthermore, the partition tree analysis identified 25 design 
point options, among 260 examined, that met the operational goals when the stealth level 
of the enemy is low. By contrast, we found only seven design point options meet the 
operational goals when the enemy stealth level is high. 

Finally, we calculated a relative cost estimate for the total system price for all 260 
design point options. The most desirable option is revealed in the relative cost efficiency 
analysis. 
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I. 


INTRODUCTION 


A. PURPOSE 

This research examines the potential effect of employing Unmanned Aerial 
Vehicles (UAV) in a detection operation in the Decisive Storm operation in Yemen. 
Results will assist decision makers in their acquisition selection of an appropriate UAV to 
support detection operations. Furthennore, this study can show how the Gulf Coalition 
Forces (GCF) participating in the Decisive Storm operations can improve border security 
and defend their operational areas against enemy incursions and reconnaissance. 

B. MOTIVATION 

Saudi Arabia leads the Gulf Coalition Force (Bahrain, United Arab Emirates, 
Qatar, and Kuwait) in executing Operation Decisive Storm, a military operation in 
Yemen against Houthi militias and their allies (Adaki 2015; Mello and Knights 2016). 
This operation defends the Yemeni government and protects northern Yemen’s and the 
Kingdom of Saudi Arabia’s borders (Al Jazeera 2016). 

Detecting the militia’s scouts, infiltrators, and terrorists before they perform their 
reconnaissance or attack is critical for defending the border and the Gulf Coalition 
personnel. Roberts (2012) stated that during the GCF operation in Yemen, the Houthi 
militias successfully gathered location information about the Emirati and Bahraini troops. 
With this information, the Houthi militias were able to fire a Soviet Tochka missile that 
killed more than 50 soldiers. Other occasions have also resulted in losses to the GCF, 
frequently because of the force’s inability to detect those infiltrators. 

This chapter provides infonnation about the Gulf Corporation Council (GCC) and 

its integrated military forces, the Peninsula Shield Forces (PSF), for which the Unmanned 

Aerial System (UAS) is a potential solution. A short background about the conflict 

environment and the threats confronting the Kingdom of Saudi Arabia’s border and on all 

other GCC members follows this discussion. This chapter lists major challenges 

in securing the Yemeni-Saudi border, describes the inadequate detection means used by 

the GCF in that region, and suggests probable solutions. 

1 



c. 


BACKGROUND 


This section provides essential information about the GCC and the conflict 
environment. It discusses the Decisive Storm objectives and challenges. This section 
further explains the current detection means used by the GCF and other possible options 
to enhance the detection operations. 

1. GCC Strategic Location and Surrounding Challenges 

The six GCC countries (Kingdom of Saudi Arabia, Kingdom of Bahrain, United 
Arab Emirates, Oman, Kuwait, and Qatar) are located in the Arabian Peninsula. 
Together, these countries form a coalition for resolving or mitigating political, economic, 
and security issues. They have merged their security policies and defense capabilities to 
form the Peninsula Shield Forces (PSF). The GCC states share borders with Yemen in the 
southern part of the peninsula and Iraq and Jordan in the northern part (Figure 1). 


Iraq 


State of Kuwait 


Kingdom of Bahrain 


Kingdom of Saudi Arabia 

9 


9 

sift of qS6P“ 


Th* is Knar Guf* 


United Arab Emirates^ 


TMlOS«l 
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Figure 1. The Gulf Corporation Council States. Adapted from 
Secretariat General of the GCC (2017). 


The GCC countries are surrounded by countries that have internal conflicts and 

elements of destabilization making it critical for the GCC states to protect their borders 

against terrorists, spies, members of criminal organizations, and illegal immigrants. 
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Investing in detection and intelligence equipment is essential to maximize border 
protection. In particular, rapid technological changes and evolving concealment 
techniques used by terrorists demand new means for detection and intelligence gathering. 

2. Decisive Storm Objectives 

In March 2015, Saudi Arabia formed the Gulf Coalition to defeat the Houthi 
militias and those loyal to the former president of Yemen and to restore legitimacy of the 
Yemeni government. This quest was based on United Nations (UN) Resolution number 
2216 (UN Security Council 2015). 

Along with Gulf States (except Oman), other nations have joined the military 
operations (Figure 2). The United States has joined the operation and provides logistical 
and intelligence support to the GCF during the military operation in Yemen (Alshabeeb 
2015). Graham (2016) reports that United States’ (U.S.) forces as well as the United 
Kingdom C4I (command, control, communications, computers, and intelligence) 
personnel have been deployed in the operation centers in command and control to 
provide assistance in controlling air strikes and detecting threats. 


THE DECISIVE STORM’COALITION 

Saudi Arabia, along with 9 other countries, launched a military operation against the Houthi militia. 

Gult States - with the exception ot Oman - are providing military support. 


Yemen's airspace as a 
"restricted area" altar 
Soud King Salman bci 
AOduiaziz ordered the 
gitslrikes. The air 
campaign has already 
resulted in lire 
eimination of several 
Houthi leaders. 

Saucf Defense Minister 
Prince Mohammed 
bin Salman hud 
waned Ahmed Ai 
ADdulah Saleh, me 
son ol Yemen's lormer 
President Ali Abdulah 
Sdeh, against 
advancing toward 
Aden. 

nre Houtt* have 
joined torce with the 
loyalists of former 
President Saleh in llteii 
offensive to take 
control ot Yemen 



Figure 2. Decisive Storm Military Coalition Forces. 

Source: Alshabeeb (2015). 
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3. 


Conflict Environment 


After the Iraq war in 2003 and the Arab Spring, Iraq and Yemen suffered from an 
unstable security environment, as well as a weak economy (Cordesman, 2015). Both 
countries have borders adjacent to GCC nations. Weak security situations and availability 
of arms and weapons have nurtured a fertile enviromnent for terrorist organizations to start 
their operations. Terrorist organizations increase the threat of smuggling and drug 
trafficking as well as migrating terrorist members into the GCC countries to vandalize 
civilian- and government-owned properties. Michael Knights (2006) has pointed to this 
issue, stating that “contiguous borders with Yemen and Iraq—both key theaters of 
operation for jihadists—make Saudi Arabia a critical transshipment point for weaponry and 
jihadists engaged in multidirectional flow of personnel and equipment throughout the 
GCC” (35). He argues that the threat posed by terrorists from Yemen and Iraq necessitates 
enhancing security for GCC countries that share borders with Yemen and Iraq. 

4. Decisive Storm Challenges 

Detecting infiltrators is the key to success for the GCF if they want to ensure border 
security. Additionally, securing the operational area against enemy incursion and 
reconnaissance is critical. There are many challenges that limit the detection process in that 
region. 


a. Technological Challenge 

The first challenge is detecting the militia’s scouts or other infiltrators, which is 
critical for securing the border and protecting the Coalition Forces. Electronic detection 
equipment such as ground radars is inadequate because of the size of the operational area 
and the characteristics of the terrain that interfere with such equipment. This leaves the 
GCF vulnerable to the Houthi militias, who have been sending their scouts across the 
border to collect reconnaissance information about the Coalition troops’ locations to attack 
and cause damage. 
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b. Terrorism Challenge 

The GCF must contend with terrorist organizations, such as Al-Qaeda in the 
Arabian Peninsula (AQAP) or the Islamic State in Iraq and Syria (ISIS or Daesh) located 
in Yemen that are trying to enter the Arabian Peninsula to cause havoc. Saeed Al-Batati 
and Kareem Fahim (2015) report that terrorist organizations located in Yemen, such as 
AQAP and ISIS, are trying to take advantage of the instability and loss of border control 
to enter the Arabian Peninsula and carry out their missions. Al-Batati (2016) further 
explains that after the Yemeni troops loyal to the former president left the city of 
Mukkala in Yemen, they created an opportunity for terrorists. Thus, he notes, Al-Qaeda 
members promptly took over the city knowing that the GCC forces and the Coalition 
Forces were otherwise occupied in conflict with the Houthi militias. 

c. Terrain Challenge 

Another challenge for the GCF in the operation in Yemen is the geography. The 
area of operation, illustrated in Figure 3, is located in the southwest part of the Arabian 
Peninsula where hills and rugged mountains dominate. Gulf Coalition Forces are 
accustomed to working in the eastern part of the Arabian Peninsula where dry desert and 
coastlands cover most of the area where detection of infiltrators is easier (Hesham 2007; 
Witty 2001; Fryberger et al. 1984). 



The image on the right shows an enlarged image of the Yemeni-Saudi border and the 
operation area is shaded in red 


Figure 3. The Arabian Peninsula Landscape. Adapted from Nielsen (2012). 
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5. Detection and Intelligence Methods Used in Decisive Storm 
Operations Are Inadequate 

Manned aircraft patrol the border area in this military operation. They also 
provide intelligence, surveillance, and reconnaissance (ISR) over the border or around the 
operational areas where troops are located. The Gulf Coalition ground forces utilize a 
variety of methods and equipment to detect (1) terrorists trying to cross the border at 
Saudi Arabia and (2) Houthi militias gathering information undercover to attack GCF 
troops. Electronic radar sensors, night vision scopes, ground motion sensors, and 
detection monitors have proven ineffective to detect these infiltrators. While some 
manned aircraft support has proven successful, the cost associated with this method is 
prohibitive. The major reason for the failure of other methods is the rough topography of 
the operational area. 

Rough topography, the size and shape of the operational area are significant 
factors that limit the effectiveness of the GCF to prevent intrusions. These challenges 
motivate the employment of new detection technologies to bolster GCF capabilities. 

The author visualized the process by which the GCF would decide on what 
detection options are better suited for the force. Following the systems engineering 
mindset in defining the problem and suggesting a possible solution, the author used a 
multiple-criteria weighted scoring model with author derived weights and scores to 
examine the best detection method for the GCF for use in the operation area. 

Options were evaluated against each other and given a weighting based on their 
importance. Total rank score determined the most effective option to be employed by the 
GCF. Table 1 shows the four options evaluated; the UAV, with the highest score of 
15.55, is the best choice for GCF purposes. Based on this preliminary study and the 
GCF’s needs, the UAV is the focus area in this thesis; further detailed analysis of UAV 
characteristics and cost-related concerns are presented in subsequent chapters. 
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Table 1. Analysis of Detection Options for the Gulf Coalition Forces. 


Evaluation Measures 


Alternatives Score 

Alternatives Weighted Score 

(Architectural Drivers) 

Weights 

Manned 

aircraft 

Balloons 

Ground 

forces 

UAV 

Manned 

aircraft 

Balloons 

Gound 

Forces 

UAV 

Cost 

0.8 

1 

2 

3 

4 

0.8 

1.6 

2.4 

3.2 

Detection Range 

0.7 

4 

2 

1 

3 

2.8 

1.4 

0.7 

2.1 

Tough terrain employment 

0.6 

3 

1 

2 

4 

1.8 

0.6 

1.2 

2.4 

Ground forces integration 

0.7 

1 

3 

4 

2 

0.7 

2.1 

2.8 

1.4 

Mobility 

0.5 

2 

1 

4 

3 

1 

0.5 

2 

1.5 

Susceptibility 

0.65 

4 

1 

2 

3 

2.6 

0.65 

1.3 

1.95 

Operational duration 

0.75 

2 

3 

1 

4 

1.5 

2.25 

0.75 

3 






Total Weighted Score -> 

11.2 

9.1 

11.15 

15.55 






Overall Rank-> 

2 

4 

3 

1 


The motivation to employ UAV technology in the operation in Yemen is 
bolstered by anticipated success in enhancing the detection and ISR mission against 
terrorists and infiltrators, as well as by considering other factors such as the cost and 
limitations associated with other detection options. 

D. PROBLEM STATEMENT 

There is inadequate analysis to support acquisition decisions to procure UAV 
technology that can significantly improve Gulf Coalition Force capabilities to detect 
crossings or encroachments on the border. 

E. SCOPE OF THE RESEARCH 

This study can aid GCF decision makers in their acquisition decision for a 
suitable UAV to support detection missions. Furthermore, this study can show the 
importance of integrating the air surveillance UAV with ground forces and even the 
efficacy of replacing other detection techniques currently used. The study also identifies 
the UAV technology employment options to serve the region. 

This thesis analyzes the use of UAVs along with other detection methods already 
used by ground forces for the GCF. Using the UAVs in the Decisive Storm operational 
area might increase the probability of detection of those infiltrators and militia members. 
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F. RESEARCH QUESTIONS 

This study addresses the following major research question: How can a UAV 
technology assessment inform the Gulf Coalition Force’s acquisition decisions for 
improving its early warning detection capabilities in the Decisive Storm operation? 

Specifically, this research is guided by these questions: 

• To what degree can UAV technology improve the detection capability for 
the GCF to increase border security and better defend the Saudi-Yemen 
border? 

• What UAV characteristics are most important for the GCF to consider in 
procuring UAV technology? 

• What combination of UAV capabilities most effectively increases target 
detection and classification in the operational area? 

• What number of UAV systems is required to achieve a certain threshold 
for detection and enhance border security? 

• How will cost constraints influence the selection and eventual purchase of 
the ideal UAV? 

G. THESIS ORGANIZATION 

The remainder of this thesis is organized as follows: Chapter II reviews other 
related research that identifies a similar UAV role in detection as well as agent-based 
modeling studies. Chapter III explains the setup of the model and the scenarios chosen 
for the purpose of this thesis. This discussion includes brief details about agents and their 
settings as well as explanations of the terrain map and elevation maps used in this 
analysis. Additionally, Chapter III explains the design factors chosen for this analysis and 
method to explore the design of experiment space. Chapter IV analyzes the experiment 
results and reveals the best UAV characteristic options for the GCF. Chapter V concludes 
the thesis by answering the research questions to illustrate their achievability. 
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II. LITERATURE REVIEW 


This chapter provides background information on UAVs and their benefits in 
modem warfare. Next, it discusses past studies about the UAV role in border control and 
studies about modeling UAVs to design a system. Furthermore, it includes an explanation 
and discussion of the modeling and simulation techniques that influence the approach 
used in this thesis. 

A. UAV BACKGROUND 

The U.S. government is the main defense supplier for the allied Arabian Gulf 
countries (Serrano and Vats 2016). However, due to the restrictions on the sale of armed 
and unarmed UAVs that the United States Department of Defense has imposed on the 
Gulf States (Cordesman and Peacock 2015), the Gulf States lack extensive experience 
with this technology. 

A brief summary of the unmanned aerial vehicle technology will assist in the 
acquisition decision maker’s understanding of the UAV system. UAVs are a component 
of the unmanned aircraft system according to the U.S. Department of Defense (DOD) 
“UAS Road Map for the UAS from 2005 to 2030” (Cambone et al. 2005). DOD refers to 
the UAS as the whole system whose elements are used to control the unmanned aircraft. 
Examples are the vehicle, human operator, payload and other elements. This thesis limits 
its focus to the component of the UAS system highlighted in Figure 4. 
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Figure 4. Unmanned Aircraft System Components. 


UASs come in different categories for a variety of mission types. Charles 
Sulewski (2005) describes four different classification of UASs in support of the U.S. 
Army: Class 1 provides Reconnaissance, Surveillance and Target Acquisition (RSTA) 
capabilities for the platoon level; Class 2 provides the same capability as Class 1 in 
addition to target designation for the platoon and company level; and Class 3 improves 
upon Class 2 systems by adding a communication relay facility on the UAS for the 
combined arms battalion level. Finally, Class 4 UASs have the capability of flying in 
conjunction with manned aircraft during missions. Table 2 shows the latest UAS 
classifications based on DOD information. 


Table 2. Current UAS Systems Classified by U.S. Army. 

Source: United States Army (2010). 


Category 

Size 

Take-off weight 

Altitude 

Airspeed 

Group 1 

Small 

0-20 

<1200 

<100 

Group 2 

Medium 

21-55 

<3500 

<250 

Group 3 

Large 

<1320 

<18000 

<250 

Group 4 

Larger 

>1320 

<18000 

Any airspeed 

Group 5 

Largest 

>1320 

>18000 

Any airspeed 
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The UAV’s rapid technological evolution has changed its role in modern warfare. 
It is an important part of combat due to the additional advantage it adds to the forces. 

B. RESEARCH STUDIES 

Many research studies address the benefit of UAV for homeland security and 
show the major role of the UAV technology in this matter. Additionally, significant 
number of thesis research conducted by Naval Postgraduate School (NPS) students about 
modeling UAVs to design a system, depict the importance of UAV in today’s warfare. 

1. UAV Effectiveness in Border Security 

Unmanned aerial vehicles have become a key player in today’s warfare due to the 
advancement in technology. UAVs can cover a wide range of tasks depending on the 
requirement of the user. UAVs have associated cost and finances that stakeholders must 
consider relative to other procurement requirements. However, many benefits outweigh 
those costs when it comes to the security of borders against terrorists. 

Christopher Bolkom (2004) points to the UAV costs and benefits. He states that 
the UAV could address shortcomings in border security surveillance if the correct sensor 
is used; that is, a sensor capable of identifying targets precisely. Similarly, UAVs capable 
of longer flight times can influence border security as they can sustain coverage for 
longer periods. He also points out the higher cost associated with the purchase of the 
UAV in comparison with manned aircraft, that difference in cost is offset by the UAV’s 
superior endurance. Bolkom also identifies some drawbacks associated with UAVs, such 
as the high rate of accidents, and relatively lower redundancy and survivability factors in 
UAV design compared to manned aircrafts. 

Chad Haddal and Jeremiah Gertler (2010) list the strengths and weaknesses of 
deploying UAVs along the United States border. They assert that UAVs could fill the 
detection gap associated with currently used surveillance equipment. In addition, UAVs 
can act as range extenders for border surveillance when comparing UAVs with traditional 
border surveillance techniques. Nevertheless, UAVs have some weaknesses, according to 
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Haddal and Gertler, associated with cost of operation, such as their high accident rate and 
the degradation in target detection under bad weather conditions. 

2. Modeling UAVs in Battlefields 

Begum Ozcan (2013) has analyzed the use and effectiveness of UAVs to support 
the security of Turkey’s east border, adjacent to the Iraqi border, where terrorist members 
locate and travel between the two countries. In her study, she illustrates the difficulties in 
detecting terrorist members located in tough, mountainous terrain using regular detection 
equipment and manned aircraft. The UAV can be adjusted technologically to meet the 
stakeholders’ requirement for such a need, if the affecting factors can be determined. 
Ozcan built a model to explore the best technological capabilities of a UAV that can be 
used to expand coverage along the Turkey-Iraqi border. She examined possible effects 
that UAVs could have in detecting and classifying infiltrators and terrorist members 
along the border. Her study showed that the use of a UAV is very effective in the 
detection and classification of terrorist activity in this challenging region, taking into 
account the significant technical factors that affect UAV detection and classification 
performance. 

Another study conducted by Bahri Yildiz (2009) focuses on the importance of 
border protection in deterring terrorist members, human and drug trafficking, and illegal 
migration by improving the detection methods along the border. In his study, Tucson, 
Arizona, was modeled using Map Aware Non-Uniform Automata (MANA) software to 
investigate the added value of UAVs in border security along with other detection assets, 
such as ground force agents, security surveillance towers, sensors, predator UAVs, and 
communication centers. Yildiz concludes that UAVs could provide an additional benefit 
in capturing illegal infiltrators and could increase the security of the borders. According 
to the outcomes of his model, three affecting factors to improve the detection means are 
manpower of the patrolling agents, communication network infrastructures, as well as the 
UAV itself. 

Fatih Sen (2015) points to the challenges that Turkey faces protecting and 
securing the Turkey-Iraqi border and the limitation to the manned aircraft used for the 
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security of border missions. The limited fuel capacity of manned aircraft and 
geographical limitations across the border have encouraged the Turkish government to 
explore other means of detection to enhance their ability to counter terrorists in that 
region. Sen (2015) also uses the MANA software to model different scenarios that are 
possible in that region and analyzes the outcomes to test the use of UAVs and manned 
aircraft. His study outcome shows that with the use of UAVs, additional target 
identification could be observed, which can lead to protecting the borders more 
effectively against terrorist attacks. 

C. AGENT-BASED MODELING SIMULATION 

Simulation software plays a major role in the analysis of combat and supporting 
decision making, as it shapes solutions to the problem and explores numerous possible 
outcomes with low cost. Several such applications include wargaming, training, system 
overall efficiency simulators, and modeling software enables different military tactics to 
be practiced. Thomas Lucas et al. (2012) note that DOD decision makers usually use 
simulation to make decisions on acquisition programs, equipment employment choices, 
and possible tactics and procedures. 

Eric Bonabeau (2002) lists several advantages of agent-based simulation over 
other techniques. Such advantages include the ability to 

• capture emergent events that result from interactions of individual entities, 

• provide natural description of the system or closer to reality simulation, 

• provide flexibility of the system by adding or subtracting agents or 
objects. 

Simon Taylor (2014) asserts that “agent based modeling has evolved as a natural 
response to the need for complex system modeling” (3). He also mentions that agent- 
based modeling has the capability to represent a system in a sensible way, more so than 
other traditional techniques. Based on his definition of the agent-based model, each 
agent-based model should have four characteristics: 

• Each agent should have a set of attributes that define the condition of the 
agent and his actions. 
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• Relationships must define how the agents relate and interact with other 
agents and the environment in the model. 

• The environment where the agent lives and variables define how he deals 
with the environments. 

• A system consists of agents, environment, and the relationship between 
them. 

The agent-based modeling is generated using a suitable programing tool and can 
be executed through a wide range of commercially available programs, such as MANA, 
MATLAB, C++, and other programs. 

D. MAP AWARE NON-UNIFORM AUTOMATA SOFTWARE 

Map Aware Non-Uniform Automata, or MANA, is one of the simulation 
programs that incorporates agent-based modeling. MANA simulation captures the 
unexpected outcomes of interactions between agents based on their behavior in the 
simulation environment. Agents could be in fonn of a human, airplane, UAV, vehicle, 
ship, and other types of key players in the simulations. As mentioned earlier, MANA is 
an agent-based distillation model that excludes unneeded complex computations, but at 
the same time produces all necessary data required for the analysis. MANA was 
developed to explore key concepts: (1) situational awareness, (2) communications, (3) 
terrain maps, (4) waypoints, and (5) agent personality change due to events (McIntosh 
2007). 

Ozcan (2013) identifies some of the advantages of MANA. It is easy to learn in a 
short time compared with other modeling software, is user friendly, and has a simple 
graphical user interface (GUI) that enables building scenarios in a shorter period. Many 
other Naval Postgraduate School theses have used MANA to analyze UAV capabilities 
and effectiveness such as those by Arif Ipekci (2002), Sulewski (2005), Ozcan (2013), 
Sen (2015), and Whye Kin Melvin Cheang (2016). 
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III. METHODOLOGY 


A. INTRODUCTION 

To address the research questions in this study, we use a modeling and simulation 
based systems engineering approach. We use abstractions of the relevant entities in the 
operational scenario to conduct simulation experiments that produce data with the 
appropriate pedigree that enables in-depth analysis and examination to support 
acquisition decisions. This chapter starts with a discussion of the measures of 
effectiveness (MOE), followed by a scenario description for the situation in which the 
system will be placed and simulated in the MANA software. Following that is a 
discussion of significant factors that might influence the MOE. Finally, we describe the 
design of experiments that we use to efficiently and effectively explore the decision 
space. 

B. MEASURE OF EFFECTIVENESS 

John M. Green (2001) states that every system development process should have 
a performance analysis; it is essential to evaluate the system’s development within its 
resources. Part of this perfonnance analysis is the performance measurements that need 
to be carefully selected, also called the MOE. Green defines the MOEs as “the measure 
of effectiveness that provide a quantifiable benchmark against which the system concepts 
and implementation can be compared” (1). 

The measure of effectiveness should be allied with defining and achieving the 
operational concerns. This thesis embraces two MOEs that are related to the improved 
effectiveness of the GCF’s detection capabilities when UAVs are integrated with ground 
radars. Those two MOEs will aid in answering the research questions of this study. 
Simulations have red and blue agents to describe different factions in the operational 
environment. 

The first MOE is the percentage of the total red agents detected (MOE 1). This 

percentage is obtained at the end of each run. Each simulation run has 500 red agents 

trying to cross through the military restricted area. MOE 1 represents the added benefits 
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of employing a UAV in conjunction with the available ground radars. Detecting 100% of 
the red agents is the ultimate goal of the GCF commanders, but it is unrealistic. 
Therefore, we set the MOE 1 threshold to 90% of the red agents, which equates to 450 
red agents. 

The second MOE is the time it takes to detect 40% of the red agents (MOE 2). 
Detecting red agents at the early stages of the operation is important so that the GCF can 
determine the center of activity to pursue. The mountainous environment that the GCF 
must traverse to engage infiltrators makes timely detection a necessity to rally a timely 
response. As mentioned in earlier chapters, Houthi militias (red agents) are adept in 
rugged terrain and travel faster, which requires the GCF to respond quickly. The total 
simulation run time is 24 hours. We set the MOE 2 threshold to 3.55 hours or 15% of the 
total simulation time. 

C. SCENARIO DESCRIPTION 

The scenario in this model is based on the historical attacks by Houthi militias 
against the GCF that have resulted in the deaths of Saudi, Bahraini and Emirati troops. 
Houthis’ reconnaissance missions are successful due to the weak detection equipment 
available to the GCF. Additionally, Houthis have the advantage of using the rugged 
terrain to conceal their movement when gathering infonnation. 

In the scenario, blue forces represent the GCF, who try to defend against the 
enemy and detect them before they reach their goals. The GCF have radars across the 
Saudi-Yemeni border to detect infiltrators before they cross the border and reach their 
objectives. However, ground radars have a low probability of detecting the enemy 
because of the terrain and the enemy’s ability to avoid detection. 

The red forces are the Houthi militias who try to make their way to the border to 
perform reconnaissance without being detected by blue force detection equipment. As 
previously noted, there are 500 red agents in this scenario. 

We are examining the improvement in detecting infiltrators when UAVs are 
employed by the GCF. For this reason, we initially model a baseline configuration that 


16 



represents the current detection equipment used by the GCF. Later on, we add UAVs to 
examine improvement in detection. Data is collected at the end of each configuration run 
to develop an analysis to answer the thesis questions. 

1. Baseline Configuration (Ground Radars Only—Current Capabilities) 

The baseline model configuration simulates the current situation of the GCF 
method explained previously. Five hundred red agents divided into ten groups of 50 each, 
are trying to cross the border successfully without being detected in the military restricted 
area. The GCF ground radars are the only detection equipment used in this configuration. 
The baseline configuration results will be used for comparison and analysis against the 
improved configuration. 

The appearance of any agent in the restricted area is considered a target with 
which GCF must deal. It is a restricted area and presence in this region is prohibited. 
Figure 7 depicts the battlefield locations. If a red agent is detected by any of the radars, it 
is reported to the GCF and eliminated from the scenario. Figure 5 shows the locations of 
the red agents, ground radars, and borderline in the baseline configuration. 
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2. Establishing the Capabilities of the GCF Radars 

Because the number of radars used across the border by the GCF, the probability 
of detection by the radars, and the radar locations are all classified, we implement a 
reverse engineering approach to build an initial model that reasonably represents the GCF 
detection capability with baseline radars. 

Multiple combinations of the number of radars, radar ranges, and radar times 
between detection were tested in the model. These combinations resulted in a percent of 
red agents detected. To establish a baseline configuration, the author used the 
combination of value settings of the number of radars, radar ranges, and radar times 
between detections that resulted in approximately 30% of red agents being detected. This 
percentage is the author’s best estimate based on his operational experience and an 

understanding of the GCF policies. More details are provided in Chapter IV. 
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3. Improved Configuration (Ground Radars and UAVs) 

The improved configuration is similar to the baseline configuration except for the 
employment of the UAVs to increase the detection rate. UAVs will patrol the area of 
responsibility (AOR) or the restricted area to detect red agents. This configuration will 
employ different numbers of UAVs (from one to four). As the number of UAVs 
increases, their search pattern and the area of coverage change in that particular scenario. 
Results of the baseline configuration will be compared to the improved configuration to 
address the research questions. Figure 6 shows a screenshot of the improved 
configuration with four UAVs; the UAVs are circled in the figure. 



Figure 6. Improved Configuration Snap Shot with 
Four UAVs Setup. 
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4. Summary of Scenario Description 

Three different groups of entities are modeled in this thesis. These groups are red 
agents that represent the enemy; blue radars, which represent the current detection 
equipment used by the GCF; and finally, the blue UAVs, which represent the new 
detection means. There will not be any simulated combat in this study. 

a. Red Agents 

Red agents are Houthi militias and other terrorist group members trying to cross 
the border; they consist of ten groups of 50 members in each group. Their main objective 
is to reach the border and get close to the GCF. The red agents are located at the 
southern part of the operational area and are moving north towards Saudi-Yemeni 
border, at a walking speed of four kilometers per hour. 

Red agents have a concealment level that allows them to hide from blue detection 
assets. The higher the concealment level, the better the red agents are at staying away 
from detectors. In addition, they can increase their camouflage by using terrain paths, for 
instance, by traveling off roads or in mountains that have higher concealment levels than 
roads. Blue force radars along the border or the UAVs can detect red agents at any time 
during the operation. When red agents are detected, detection timing information will be 
recorded. 

b. Blue Agents 

Blue radars across the border have maximum range and detection capabilities. 
Nonetheless, some of the radars have limited detection because of the topography of the 
scenario, which may prevent blue radars from detecting red agents. Blue UAVs will 
jointly perform detection of red agents along with blue radars. Each simulation run 
represents 24 hours of the operation. 

D. DESIGN FACTORS AND EXPERIMENTAL DESIGN 

Designs of experiment (DOE) are used to examine the relationship and effects of 
different factors that might influence the outcomes of the MOE. Every factor in the DOE 
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has a range of values. The author has used analogous operational systems to estimate the 
appropriate range of values used in the DOE. In this thesis, a DOE helps to identify 
factors that have the largest effect on the MOE. The DOE further assists in reducing the 
variance of MOE estimates. 

Jack Kleijnen et al. (2005) describe the DOE as a method that permits the user to 
vary input parameters to produce multiple variations in the model in an effective way. 
The DOE technique is used to detennine the cause and effects bond, which is necessary 
for the selection of the inputs to enhance outcomes. Douglas Montgomery (2008) states 
that a design of experiment is a test where changes are made to the input variables on 
purpose, so we can observe the reasons for a change in the outcomes. 

Factors can be categorized as controllable and uncontrollable factors. Controllable 
factors are those elements of the system that can actually be changed by the manufacturer 
or operators. The maximum flight altitude or speed, for example, are controllable factors. 
Uncontrollable factors are the factors that the designer cannot change in real life but can 
change during the experiment. Enemy characteristics and numbers are some examples of 
uncontrollable factors. 

1. UAV 

The UAV factors were selected based on Group 1 to Group 3 UAV technical 
characteristics (see Table 2), since this type of UAV category fits the required mission in 
detection and border control (United States Army 2010). Other UAV groups were 
eliminated for two reasons: they fly at a strategic high altitude level that is not required 
for this research, and their operating radius is very large. 

The number of UAVs (one to four UAVs) is a design factor that will be crossed 
with the base experimental design. The UAV number factor will detennine the minimum 
number of UAVs needed to achieve the required thresholds in the operation area. 

• UAV speed is a flight performance factor that applies to the UAV. The 
UAV speed refers to the velocity the UAV travels at during the search 
mission. 
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• UAV altitude is another flight perfonnance factor that defines the height 
that the UAV can fly in the covered area. 

• Endurance is the ability of the UAV to fly without the need to refuel. It 
defines the UAV’s stamina. 

• UAV detection range is a sensor related factor that defines the maximum 
distance at which the sensor can detect targets. 

• Time between detections is a factor that sets the period between “looks” at 
a search area. It defines how the UAV detects an object in the search area. 

• Slew rate is the factor associated with how many degrees the UAV sensor 
can rotate per second. 

• Aperture is the angle width that the sensor can search at each point of the 
rotation. 

• UAV refuel time is the time it takes the UAV to be refueled and prepared 
for flight. This factor should reflect lower values, as it will have a negative 
effect on the availability of the UAV. The more time UAVs spend 
refueling, the higher their absence rate. 

2. Red Agent 

Red stealth is an uncontrollable factor. It represents the ability of the red agent to 
avoid being detected by radars or UAVs. Uncontrollable factors are introduced into 
models to reflect a real-life source of variations, which increases the credibility and 
robustness of outcomes of this study. 

3. Factor Ranges 

Factor ranges have been defined based on the possible UAV types that could be 
employed in the Decisive Stonn operation in Yemen by the GCF (United States Army 
2010). Table 3 presents the factor ranges chosen for the experiment that are defined as 
significant factors and might have impact on the outcomes of the research. All of the 
factors were chosen based on UAVs described in open sources. 


22 



Table 3. Design of Experiment Factors and Ranges. 


Agent 

DOE factor 

Min. 

Max. 

Units 

UAV 

Speed 

80 

200 

Knots 

UAV 

Altitude 

3000 

8000 

Meters 

UAV 

Maximum detection range 

1000 

4000 

Meters 

UAV 

Time between detection 

5 

20 

Seconds 

UAV 

Endurance 

1 

8 

Hours 

UAV 

Refuel time 

30 

60 

Minutes 

UAV 

Slew rate 

60 

360 

Degrees 

UAV 

Aperture 

60 

120 

Degrees 

Red forces 

Red stealth 

65 

85 

Percentage 

UAV 

Number of UAVs 

1 

4 

Discrete 


E. PREPARING MANA SOFTWARE FOR SIMULATION 

MANA software is used as the simulation modeling software due to the ease with 
which the user can create scenarios and modify agents’ properties. MANA has the ability 
to model the behaviors of the agents to a higher resolution. Another reason to use MANA 
is to take advantage of the Simulation Experiments and Efficient Designs (SEED) center 
at the Naval Postgraduate School (NPS), which offers scenario development support and 
data analysis, as well as a power-computing cluster that can perfonn complex and 
multiple factor levels simulations. 

1. Data Entry and Control 

A database is created in MANA to store entities’ values required for the scenario. 
These values control the behavior of and set the limitations for entities in the simulation. 
MANA has the ability to store different types of maps to define the agent’s movement in 
the simulation. 

2. Battlefield 

An area of 90 x 90 km along the Saudi-Yemeni border was chosen to represent 
the operational area. This area was modeled in MANA, with relevant entities, to 
represent the GCF military operations in Yemen. The battlefield area is a restricted 
military zone and divided into two sections. The northern part is the Kingdom of Saudi 
Arabia, where the 
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GCF are located. The southern part is a rugged area where Houthi militias and terrorist 
group members make their way towards the border. As previously mentioned, the 
military restricted area excludes not only military personnel, but also civilians or neutral 
personnel. Any detection in this area will be considered a target that the GCF will treat as 
a threat. 

The border is defended by multiple radar systems to detect infiltrators and report 
the detection back to the GCF operation centers to deal with the infiltrators. Figure 7 
represents the battlefield agent’s locations and the border area. 



Figure 7. Battlefield Description and Entities Locations. 


3. Map Construction 

MANA can store elevation, terrain, and background maps to define the agent 
movement in the scenario. The difference in elevation heights can act as a concealment 
factor for the agent that increases the stealth of the agent, or restrict the movement of the 
agent towards its goal. It also plays a role in the line of sight (LOS) factor for the sensors 
that may delay the detection time. 
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The terrain features map was constructed in MANA to reference the elevation 
areas. It consists of three different levels: mountains, off roads, and valleys. To ensure the 
ground radars and UAV sensors operate properly, it is imperative to define these levels in 
the model. Figure 8 shows the terrain map with the terrain characteristics levels of going, 
cover, and concealment, as well as the terrain surface levels of road, mountain, and off¬ 
road. “Going” refers to the level of movement; “Conceal” is the visibility level; and 
“Cover” is the protection level the agent could have. Those three levels range from zero 
to one. Zero means “No” and one means “Full.” For example, when an agent uses the 
“Road,” he has ease of movement, but no cover or concealment against detection. While 
in the mountains, the agent has a high level of concealment or a good chance to avoid 
detection, but difficulty in movement. 



Figure 8. Terrain Features Map and Parameters. 
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F. DESIGN OF EXPERIMENT 

Examining all factors listed in Table 3 requires a time-efficient design of 
experiment technique to cover all factors and to effectively and uniformly sample the 
design space. A full factorial experiment is one option that requires an extended period to 
complete. A full factorial design for this thesis will produce 170 million design points, 
which would require more than five years to process if each run took one second to 
complete. While the author desires a thorough exploration of design space for this 
research to assure discovering important insights, this amount of time is impractical. 
Susan Sanchez and Hong Wan (2015) explain that experiments should be designed in a 
keen manner by using a well-organized space filling design, which this research 
accomplishes, as the following discussion describes. 

1. Nearly Orthogonal Latin Hypercube Design Method 

In lieu of a full factorial design, we are going to use the nearly orthogonal Latin 
hypercube (NOLH) 65-design point method developed by Thomas Cioppa and Thomas 
Lucas (2007). NOLH design is a method used to combine orthogonal and uniform 
designs to create efficient, uncorrelated, fully space-filling experiments (Cioppa and 
Lucas, 2007). It significantly decreases the number of design points (DP) in the 
experiment compared to a full factorial design. For these reasons, we chose an NOLH 
design. This type of design also works best when design factors are continuous values 
such as those listed in Table 3. 

An NOLH spreadsheet created by the NPS SEED center (n.d.) was used to 
generate a unique combination of factor levels for this thesis. A 65-run NOLH design to 
explore the factors in Table 3 provides good coverage of the possible combinations in the 
input space. Crossing the 65 DP with the number of UAVs in the improved configuration 
yields 260 different design points (see Appendix A). 

The scatter plot in Figure 9 shows the pairwise plots between factors in the 
experimental design. It shows the space-filling attributes of the NOLH in the two- 
dimensional space. If it were possible to create plots for all other dimensions, they would 
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present similar graphics. They depict a design that effectively and efficiently samples the 
problem space. 



Figure 9. Scatterplot Matrix All Factors. 


2. Number of Replications 

Each design point has to be replicated to reduce the variability of the estimate of 
the MOE’s outcome. Replication deals with the random errors in the experiments. For the 
purpose of this thesis, we have selected a 95% confidence interval of the estimated mean 
for the MOEs. This value of confidence interval was chosen to create the desired interval 
width of approximately three agents for MOE 1 and approximately eight minutes for 
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MOE 2. From the operations perspective, the author asserts that these are within the 
margins that the GCF can effectively operate. Therefore, determining the required 
number of run replications is important if we are to reach the desired degree of precision 
in the results. 

We determined the number of replications according to the formula that takes into 
account our estimate of the standard deviation (a) and our chosen level of alpha (a = 
0.05). Z is the variant corresponding to half of alpha, based on the standard normal 
distribution. The desired width of the confidence interval for a probability is w. 



n = 


V w J 


Forty replications per design point were sufficient for both MOEs. MOE1 needed 
33 replications while MOE2 required 40 replications. Therefore, we chose to run each 
design point 40 times. Additionally, this number of replications allows the experiment to 
finish in a reasonable time. The mean of each 40 replications is ultimately summarized 
and used for the data analysis (see Appendix B). 
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IV. DATA ANALYSIS 


This chapter contains results answering the research questions and illustrating 
how UAV technology can improve the detection capability for the GCF. Sensitivity 
analysis results are used to examine the model and detennine whether the system is 
sensitive to scenario parameters such as the specific number of red agents trying to cross 
the border. Next, a baseline configuration analysis demonstrates the percentage of 
detected red agents the current system provides in the operation. Third, scenario analysis 
and comparison with the baseline results test whether UAVs will provide an additional 
percentage of red agent detection. Fourth, a regression analysis allows us to understand 
which factors are significant. Finally, relative cost and benefit analyses provide an 
understanding of the relative cost of various options. 

A. SENSITIVITY ANALYSIS TO RED AGENT SCENARIO 

The purpose of the sensitivity analysis is to examine the consistency of the model 
outcomes with different attack scenarios in place. Figure 10 shows the distribution of the 
total number of red agents detected, percentage of red agents detected, and time to detect 
40% of red agent using the improved configuration with four UAVs. Two different 
quantities of red agents, 50 and 500, have been used to compare the distribution between 
the two formations. 

Results show that the average number of red agents detected using the 500-agent 
formation equals 406, while the 50-agent formation average scored 41. Second, the 
average percentage of red agents detected for the 500 fonnation was 81% with a 1% 
increase in the 50-agent formation. Finally, the average time to detect 40% of the red 
agents was 6.8 hours for the 500-agent formation, while the 50-agent formation took 
seven hours. 

Results show that distribution and summary statistics are very similar for both 
scenario configurations and the outcomes are not contingent on a specific scenario or a 
certain number of red agents. These statistics increase the author’s confidence in the 
model results and potential biases in the data. 
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Figure 10. Sensitivity Model Analysis. 


B. RESULTS OF BASELINE CONFIGURATION (GROUND RADARS 

ONLY) 

1. Desired Percentage of Red Agents Detected 

To define the GCF baseline capability, we are trying to find the right combination 
of ground radars such that we detect only 30% of the red agents. Three radar factors are 
examined using an NOLH method to determine the combination of the blue radar factors 
that can achieve the desired percentage of red agents: Number of radars, Maximum 
detection range, and Time between detection. 

Sixty-five different combinations across the three factors have resulted in the 
material depicted in Appendix C. The results show that combinations with a low number 
of radar stations (from four to six) were not able to provide the desired percentage of red 
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agents detected. Therefore, they are disregarded. Combinations with seven or eight radars 
were able to generate results approximating the desired percentage of red agents detected. 

The highlighted design factor (No# of radars=8, Time b/w detection=108, and 
max range of 5625) came closest to the desired baseline percentage of red agent 
detection, registering a score of 30.35% (Table 4). We have rounded them up to (No# of 
radars=8, Time b/w detection=100 seconds, and max range of 6000 meter) to ensure it 
will generate detection of at least 30% of the red agents. Among all other possible 
combinations, we use those values as the baseline for this study because those values 
seem to have the characteristics of operational understanding of the situation. 


Table 4. Blue Radar Design Points for the Desired Percentage of Red Agent 

Detections (7 and 8 Radars Only). 


NumBlueRadars 

BlueRadarRng 

BlueRadarAvgTimeBetDet 

Std Dev(Alleg2Cas.Red.) 

Mean (Percentage RedDetected) 

7 

6313 

93 

8.859443752 

0.3537 

7 

5750 

83 

8.208813275 

0.348 

8 

5188 

68 

7.361542481 

0.3375 

8 

6063 

101 

8.230493645 

0.3349 

7 

7063 

115 

9.272201354 

0.33405 

8 

6625 

116 

8.738215508 

0.3301 

7 

5000 

66 

9.185237599 

0.32825 

7 

5563 

94 

7.445141248 

0.31965 

8 

5625 

108 

8.619447302 

0.3035 

8 

5438 

117 

8.556898552 

0.2872 

7 

4813 

84 

8.318615208 

0.28615 

7 

4125 

61 

9.106070109 

0.2849 

7 

4750 

92 

9.448884075 

0.28095 

7 

4188 

82 

9.508397099 

0.25945 


2. Baseline MOE 1 Percentage of Total Red Agents Detected 

Figure 11 represents the distribution of the percentage of red agents detected for 
the selected baseline values. The detection percentage was attained by averaging the 
outcomes of 100 repetitions of the baseline design point. An average of 32.19% of red 
agent ± 2.1% was detected at a 95% Confidence Interval (Cl), which is lower than the 
required MOE 1 threshold of 90% of red agents detected. 
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Figure 11. Distribution and Summary Statistics for 
Baseline Configuration (MOE 1). 


C. RESULTS OF THE IMPROVED CONFIGURATION (GROUND RADARS 

WITH ONE OR MORE UAVS) 

1. MOE 1: Percentage of Total Red Agents Detected 

Figure 12 represents the distributions of the percentage of red agents detected 
when the ground radars, along with the one or more UAVs, are implemented in the 
improved configuration. An average of 81.24% of red agents ± 1% was detected at a 95% 
Cl, which is less than the required MOE 1 threshold of 90% of red agents detected. 

The implementation of UAVs in the operation for the GCF shows an increase of 
detection by almost 49%. The important factors that led to this increase are discussed in 
later sections. 
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Figure 12. Distribution and Summary Statistics for 
Improved Configuration (MOE 1). 


2. MOE 2: Time It Takes to Detect 40% of the Red Agents 

Figure 13 shows the total time it takes to detect 40% of the red agents in the 
operation. Adding the UAVs will benefit the GCF by speeding up the process of 
detecting the red agents, allowing GCF to determine the center of activity to pursue. This 
will enhance the probability of successfully securing the border. 
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Figure 13. Distribution and Summary Statistics for 
Improved Configuration (MOE 2). 


The data distribution for MOE2 shows a non-symmetrical distribution (Figure 
13). An average time of 6.8 hours to detect 40% of the red agents has been achieved. This 
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average mean value lands around the 65th percentile in the distribution, which means 
35% of the data can detect the red agents in shorter periods. 

The introduction of the UAVs, along with the available ground radars, has 
apparently improved the speed of the detection in comparison with the baseline 
configuration, but it still did not meet the threshold value of the MOE 2 (3.55 hours). 

3. Conclusion for the Improved Configuration 

The introduction of the UAVs with their current capabilities, along with the 
available ground radars, has significantly improved the percentage of red agent detection 
and the time to detect them, but it did not achieve the required threshold for either of the 
MOEs. This means that there should be further investigation into what improvement in 
UAV capabilities (United States Anny 2010) will increase the percentage and speed of 
detection. This investigation will aid the GCF’s decision in what specific capabilities they 
should invest. 

D. EXAMINATION OF THE UAV’S SIGNIFICANT CAPABILITIES 

We claim that a specific number of UAVs with certain capabilities will achieve 
the required threshold for both MOEs for the GCF to accomplish mission success. The 
next section examines the possible combination of UAV capabilities that could succeed. 

Introducing the UAV technology to support the detection for the GCF has shown 
apparent improvement, but it did not reach the specified operational goals. Thus, the next 
section considers the method of determining what particular UAV characteristics are 
most valuable. First, we plot the correlation between the two selected MOEs to better 
observe whether one or more design points can satisfy both MOEs. Second, we examine 
the significant factors that have the greatest impact on the UAV through regression 
analysis. Third, we use a partition tree analysis to identify the design point options that 
can meet both MOEs. Finally, we perform a relative cost and benefit analysis to discover 
the desirable design point option. 
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1 . 


Correlation between MOEs 


The two MOEs are correlated when shown graphically (Figure 14). The graphical 
presentation helps to identify possible design points that could satisfy the operational 
requirements for the GCF. Figure 14 shows a scatter plot between the two MOEs with 
their threshold levels and possible DPs that can achieve mission success. As shown in 
Figure 14, multiple DPs fall within the operational goal target, represented by the shaded 
area. 



1 3 5 7 9 11 13 15 17 

MOE2 Time to Detect 40% of Red Agents 


Figure 14. Correlation between MOEs. 


2. Significant Factors Selection 

Identifying the most significant factors of the UAV will help the GCF identify the 
UAV best suited for the required detection mission. Examining the factors will help in 
focusing on only those factors related to improving the detection capability of the UAV. 
We use regression analysis to discover the significant factors for the UAV. 
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a. MOE1 Significant Factors 

A 10-factor stepwise linear regression model results in an adjusted R square value 
of 0.68, which indicates that the regression model developed is adequate to predict the 
behavior of MOE 1. The stepwise regression analysis examines how much each of those 
factors can add to the description of the MOE 1, which will identify the important factors. 
This first step screens all significant factors for MOE 1. We do not consider interaction 
tenns in this regression model. Figure 15 lists all the factors with the largest effect on 
MOE 1. 


Sorted Parameter Estimates 


Term 

Estimate 

Std Error 

Red Stealth 

-0.00847 

0.000602 

UAVMaxDetRng 

5.6129e-5 

0.000004 

NumUAVs 

0.0327168 

0.003152 

UAVAperture 

0.0016761 

0.000201 

UAVEndurance.hrs. 

0.0078887 

0.001703 

U AVA Ititud e_Def a u It 

-3.593e-6 

2.405e-6 

UAVMaxTimeBtwDet 

-0.000911 

0.000798 

UAVSIewRate 

-2.317e-5 

4.012e-5 

U A VSpeed_Def a u It 

-2.948e-5 

0.0001 

U A VRef u elTime. m i n . 

3.11 34e-5 

0.000399 



Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

0.1365 

0.2545 

0.5641 

0.7690 

0.9379 


Figure 15. Significant Factors for MOE 1. 


The significance level (a) was selected to be 0.05, which is the limit to reject or 
fail to reject a factor. The t Ratio value is a test statistic for the hypothesis test to evaluate 
whether the estimate for that factor is significant. Patrick Runkel (2016) in his blog 
section T & P explains that “t-value measures the size of the difference relative to the 
variation in the sample data. It is the calculated difference represented in units of standard 
error.” The greater the t-value, the greater the evidence to reject the null hypothesis to 
which the associated population parameter is plausibly equal. The null hypothesis is that 
the true value of the coefficient related to the factor is 0; indicating that it has no 
influence on the MOE. 
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The estimate value in the second column is the amount of change to MOE 1 per 
one unit change in the factor-the estimated coefficient. For example, the red stealth 
estimate has a negative value of (-0.00847) because as red stealth increases one unit, the 
percentage of red agents detected decreases by this value. Another case is the detection 
range factor. It can be explained as: for every 1-meter increase in detection range, MOE 1 
increases 5.6e-5 percent. 

A second order stepwise regression analysis was performed to find significant 
factors that interact with other factors. It results in an adjusted R square of 0.73, which 
indicates that the regression model developed is adequate to predict the behavior of MOE 
1. Figure 16 shows the significant factors of the second order with their associated values. 
Five factors were significant as shown in the Prediction Profiler: Red Stealth, UAV 
Maximum detection range, Number of UAVs, UAV Aperture, and the UAV Endurance. 


Sorted Parameter Estimates 

Term Estimate Std Error t 

RedStealth -0.008464 0.000551 - 

UAVMaxDetRng 5.6055e-5 3.674e-6 

NumUAVs 0.0327168 0.00289 

UAVAperture 0.0016795 0.000184 

(NumUAVs-2.5)*(NumUAVs-2.5) -0.018226 0.003231 

UAVEndurance.hrs. 0.0080077 0.001558 

(UAVAperture-90.0308)*(RedStealth-75.0308) 0.0001045 3.25e-5 

(NumUAVs-2.5)*(RedStealth-75.0308) 0.0013708 0.000493 

Effect Details 

» Prediction Profiler 



Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

0.0015’ 

0.0058* 



RedStealth NumUAVs UAVMaxDetRng UAVAperture UAVEndurance.hrs 


Figure 16. Second Order Stepwise Linear Regression for MOE 1. 


The Prediction Profiler shows an interesting result: the Number of UAVs factor 

(as a main factor) shows that increasing the number of UAVs beyond three provides 

marginal improvement to MOE 1. This could be because three UAV sets saturate the 
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operational area and we do not need an extra UAV set. This factor requires further study 
on route development and sensor capabilities, but that is beyond the scope of this thesis. 

Figure 17 shows a matrix of interaction between the factors. A non-parallel line 
gives an indication of possible interaction between the factors and their effect on that 
MOE. For example, the highlighted interaction between the UAV Aperture and Red 
Stealth factors can be explained as follows: when Red Stealth is low (Value 65), 
changing the UAV Aperture (from 60 to 130 degrees) has minimal effect on MOE 1. By 
contrast, when Red Stealth is high (Value 85), increasing UAV Aperture (from 60 to 130 
degrees) has larger impact on MOE 1. 


Interaction Profiles 
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Figure 17. Factor Interaction MOE 1. 


b. MOE 2 Significant Factors 

Another 10-factor linear regression model was conducted for MOE 2 that results 
in an adjusted R square of 0.80, which indicates that the regression model is adequate to 
predict the behavior of MOE 2. With the same significance level selected for MOE 1 of 
0.05, Figure 18 shows, in order of importance, the significant factors that have an impact 
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on MOE 2. As seen in Figure 18, five factors have an impact on MOE 2. The Estimate 
value for the number of UAVs factor shows a negative value, which means that as the 
number of UAVs increases, the time it takes to detect 40% of the red agents decreases. 
Similarly, in the UAV endurance factor, the greater the UAV endurance means the less 
time we need to detect 40% of the red agents. This is because the UAV will not waste 
time when refueling on the ground and will spend more time in the air. The Red Stealth 
factor, on the other hand, shows a positive value, which reflects the following: as the red 
stealth level increases, the time it takes to detect the red agents increases as well. 


Sorted Parameter Estimates 


Term 

NumUAVs 

UAVMaxDetRng 

Red Stealth 

UAVAperture 

UAVEndurance.hrs. 

UAVAItitude_Defau It 

UAVMaxTimeBtwDet 

UAVSIewRate 

U A VRef u elTi me. m i n . 

U A VSpeed_Def a u It 


Estimate 

-1.858138 

-0.001803 

0.1952902 

-0.05267 

-0.306637 

0.0001244 

0.0287277 

0.0007187 

-0.004073 

0.0003005 


Std Error 

0.084219 

0.000107 

0.016078 

0.005364 

0.045507 

6.427e-5 

0.021321 

0.001072 

0.010664 

0.002679 


t Ratio 



Prob>|t| 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

<. 0001 * 

0.0541 

0.1791 

0.5032 

0.7028 

0.9108 


Figure 18. Significant Factors for MOE 2. 


Performing a second order stepwise regression model has resulted in a higher 
adjusted R square value of 0.82 and an additional factor that might have an impact on 
MOE 2. Figure 19 illustrates the significant factors of the second order with their 
associated values. 
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J Sorted Parameter Estimates 

Term Estimate Std Error t Ratio 


NumUAVs 

UAVMaxDetRng 

RedStealth 

UAVAperture 

UAVEndurance.hrs. 

(UAVSpeed_Default-140.062)*(UAVApertu re-90,0308) 
(NumUAVs-2.5)’(NumUAVs-2.5) 
(UAVAperture-90.0308)*(RedStealth-75.0308) 
UAVSpeed_Default 


-1.858138 0,079822 -23 28 
-0.001801 0.000101 -17.74 
0.1950923 0.015232 12.81 
-0.052836 0.005084 -1039 
-0.310894 0.043021 -7.23 

-0.000521 0.000143 -3.64 

0.3197308 0.089243 3.58 

-0.002841 0.000901 -3.15 

0.0001853 0.002538 0.07 



Prob> |t| 

<. 0001 * 

<. 0001 * 

<■ 0001 * 

<. 0001 * 

<. 0001 * 

0.0003* 

0.0004* 

0.0018* 

0.9419 


Effect Details 


A » Prediction Profiler 
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Figure 19. Second Order Stepwise Linear Regression MOE 2. 


The UAV Speed as a main factor has no effect on MOE 2 as shown in the 
Prediction Profiler (almost flat curve line), but it will have an effect when interacting 
with other factors (Figure 20). The interaction matrix for the MOE 2 significant factors 
shows an interaction between the UAV Speed factor and the UAV Aperture factor over 
MOE 2. When the Speed of the UAV is high (value = 200), increasing the UAV Aperture 
(from 60 to 130 degrees) has a larger impact on MOE 2. 
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Interaction Profiles 
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Figure 20. Factor Interaction MOE 2. 


As a result of this interesting finding about the interaction between factors, we 
decide to use the UAV Speed as a significant factor. Therefore, a total of six factors were 
significant for MOE 2: Red Stealth, UAV Maximum detection range, Number of UAVs, 
UAV Aperture, UAV Endurance, and UAV Speed. 

c. Major Significant Factors 

While MOE 2 has the UAV Speed as an additional significant factor that 
influences it, the two MOEs have five of the factors in common. Thus, a total of six 
factors have a significant effect on the UAV’s ability to improve GCF’s capability to 
detect and respond to infiltrators. 
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3. 


Design Points that Meet Both MOEs 


We plotted the correlation between the MOEs in Figure 14 and highlighted the 
intersection area of design points that meet the threshold level of both MOEs. A total of a 
32 design points met the criteria and are used for further analysis (Table 5). 


Table 5. Design Points that Meet MOE 1 and MOE 2. 


No# 

DP 

NumUAV 

UAVSpeed 

UAVAItit 

UAVMaxDetecRange 

TimeB/wDetec 

Endurance 

RefulTime 

SlewRate 

Aperture 

RedStealth 

1 

79 

2 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

2 

106 

2 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

3 

110 

2 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

4 

140 

3 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

5 

141 

3 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

6 

144 

3 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

7 

150 

3 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

8 

159 

3 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

9 

163 

3 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

10 

171 

3 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

11 

172 

3 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

12 

175 

3 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

13 

176 

3 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

14 

201 

4 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

15 

205 

4 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

16 

206 

4 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

17 

209 

4 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

18 

211 

4 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

19 

212 

4 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

20 

215 

4 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

21 

221 

4 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

22 

224 

4 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

23 

226 

4 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

24 

227 

4 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

25 

228 

4 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

26 

229 

4 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

27 

233 

4 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

28 

236 

4 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

29 

237 

4 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

30 

240 

4 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

31 

241 

4 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

32 

246 

4 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 


4. Partition Tree Analysis 

We use a partition tree analysis to find the most consistent or predictable way to 
satisfy both MOEs. This method enables us to explain the predicted value through a 
multiple combination of factors. The partition tree moves from top to bottom, searching 
for possible splits between factors, and it indicates the best split, based on greater values. 
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The graphical tree structure provides an easy way to understand and interpret data. It 
allows for discovering a correlation between factors in the data. Readers should note that 
every design point (DP) is a mixture of different factor levels (refer to Table 5), and each 
DP is one UAV possible option. 

a. Partition Tree Analysis for Both MOEs 

To improve the detection capability for the GCF and increase border security, we 
aim to meet both MOE thresholds. As discussed in the Design Point section, 32 DPs out 
of 260 satisfied the criteria of both MOEs. 

The following description is a simplified version of the tree analysis on which 
factors contribute most to satisfying both MOEs. For more detailed analysis with exact 
values, please refer to Appendix D. 

The partition tree in Figure 21 represents the breakdown of the overall (260) 
design points. The top box contains the DPs that met both MOEs (32 DPs) marked with 
“YES.” The first split appears on the red stealth level. Splitting the tree with the Red 
Stealth factor can assist the GCF in determining the ideal UAV in extreme cases for 
detecting red agents when they are most and least capable of being detected. 



UAV>=3 
Count 48 

"YES" 22 


UAV<3 


Count 

48 

"YES" 

3 


UAV>=4 


Count 

41 

"YES" 

6 


UAV<4 


Count 

123 

"YES" 

1 


Figure 21. Partition Tree for MOEs. 
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As the split on this level indicates, the Low Red Stealth factor has 96 DPs among 
the 260, but only 25 of them can achieve both MOEs. The High Red Stealth factor has 
164 DPs out of 260, but only seven can meet the MOEs’ criteria. 

We further split the tree to see how the UAV factors contribute to the success in 
the low and high red stealth cases. The next split under the low stealth cases is based on 
the number of UAVs, which splits evenly: 48 DPs on both sides. The “YES” criteria was 
achieved 22 times when the number of UAVs was three or more, while it was achieved 
only three times when UAVs were fewer than three. 

The high red stealth path shows seven possible DPs that can achieve both MOEs. 
Only one DP can possibly meet both MOEs with less than four UAVs. The other six DPs 
require four UAVs. 

We can further split the tree to better analyze the data and better understand which 
factor will have an effect on the achievement of the MOE thresholds (see Appendix D). 

b. Partition Tree Outcomes 

Solutions using more UAVs tend to give better results or more options to achieve 
the MOEs’ threshold, but the marginal improvement after the third UAV set does not 
appear to justify the cost. In the next section, however, we examine the benefit of an 
additional UAV in comparison to relative costs. 

There are only a few design point options to examine when the enemy is highly 
stealthy. Seven DPs have achieved the threshold for both MOEs in the high red stealth 
configuration. One of those DPs can meet both MOE thresholds with less than four 
UAVs. 

In the other scenario when the enemy is less stealthy, 25 design point options 
achieved the threshold for both MOEs. Using at least three UAV sets appears to provide 
more options to detect at least 90% of the red agents in 3.55 hours or less. The partition 
tree in Figure 21 shows three design point options that meet both MOEs’ thresholds using 
fewer than three UAV sets. Further experimentation follows in this thesis to show which 
DP option is the most cost-effective combination. 
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5. 


UAV Capability and Relative Cost Analysis Tradeoff 


Following Sangbum Kim’s (2017) approach to developing a relative cost analysis 
will help identify the best UAV capabilities at the most effective cost. Assigning each 
UAV capability with a value assists in computing each design point’s final relative cost. 
The optimum design point should have a high MOE value and lowest relative cost. 
However, a UAV’s component cost breakdown is unavailable and hard to obtain within 
the time frame of this thesis. Therefore, we create a relative cost scale for each UAV 
capability in terms of the user priorities to develop an estimate cost for each DP. A 
comparison between the DP final relative cost and its MOE value will aid in defining the 
option we desire to pursue. 

To calculate the expected cost for each design option, we need a reference UAV 
model upon which to base our calculation. As mentioned in Chapter III, a UAV from 
Group 1 to Group 3 was used to set the factor ranges in this study. The Raven RQ-11 
UAV (Figure 22) used by the U.S. military (Kasper 2014) falls into this category and is 
used as a cost reference for this study. 



Figure 22. Raven RQ-11 UAV. Source: AeroVironment.inc (2017). 


The estimated cost of the Raven RQ-11 is $300,000 USD for a set of three UAVs 
with the two ground control stations and its supporting equipment (Kasper 2014). The 
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price of one UAV vehicle is estimated at $34,000. The author has derived a price 
breakdown of the Raven RQ-11 system (Table 6). 


Table 6. Author-derived Raven RQ-11 Cost Breakdown. 


Raven RQ-11 Cost 

$ 300,000 

Vehicle Cost 

$ 34,000 

Vehicle Cost x 3 (Three vehicles) 

$ 102,000 

GCS (x 2) + Supporting equipment 

$ 198,000 


a. Factor Relative Cost Scale 

The cost of each design point is a product of the UAV capability factor cost. 
Therefore, we need to compute the relative cost of the eight UAV capability factors. The 
Analytic Hierarchy Process (AHP) can aid in calculating UAV factor weights for this 
study. 

The AHP process runs a pairwise comparison between the factors based on which 
factor has more importance in terms of UAV performance (Table 7). The more important 
the factor in regards to the UAV performance, the higher the relative cost it will have. 


Table 7. UAV Eight-Capability Factor Pairwise Comparison. 



UAVSpeed 

UAVAItit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefuelTime 

SlewRate 

Aperture 

UAVSpeed 

1 

1/3 

1/4 

3 

1/5 

2 

1/2 

1/6 

UAVAItit 

3 

1 

1/2 

5 

1/3 

4 

2 

1/4 

UAVMaxDetecRa nge 

4 

2 

1 

6 

1/2 

5 

3 

1/3 

TimeBwDetec 

1/3 

1/5 

1/6 

1 

1/7 

1/2 

1/4 

1/8 

Endurance 

5 

3 

2 

7 

1 

6 

4 

1/2 

RefuelTime 

1/2 

1/4 

1/5 

2 

1/6 

1 

1/3 

1/7 

SlewRate 

2 

1/2 

1/3 

4 

1/4 

3 

1 

1/5 

Aperture 

6 

4 

3 

8 

2 

7 

5 

1 


Normally, a user compares a single factor against other factors and assigns a score 
depending on how important it is (from one to nine, where one is equally preferred and 
nine is extremely preferred) (Bodin and Gass 2003). For this study, however, the factor’s 
values for importance are informed by the regression analysis results in the data 
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correlation section. The Estimate column in the regression analysis shows how much 
change in the MOE can be caused by the one-unit change in the factor. For example, in 
the MOE 1 regression analysis, the UAV aperture factor has a greater effect on the 
percentage of detected red agents over UAV endurance, and hence, the UAV aperture is 
preferred over the UAV endurance factor and receives a higher value. 

Data in Table 7 represent the importance value when comparing the row factor to 
the column factor. An integer (1 to 9) means the factor in the row is more preferred to the 
factor in the column by the integer amount. A fraction of the integer indicates that the 
factor in the row is less preferred than the factor in the column. 

The next step is to normalize the data in the matrix shown in Table 7 to obtain the 
factors’ weight scores. First, we sum each of the pairwise comparison columns; then, we 
divide each entry cell in the matrix by its column sum value (Bodin & Gass 2003). The 
average value in each row of the normalized matrix serves as the weight value for the 
UAV capability factor. Table 8 presents the score results, which are highlighted in 
orange. 


Table 8. UAV Eight Factors’ Weight Scores. 


Normalizing and Comparisons 



UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Score 

UAVSpeed 

0.05 

0.03 

0.03 

0.08 

0.04 

0.07 

0.03 

0.06 

0.05 

UAVAltit 

0.14 

0.09 

0.07 

0.14 

0.07 

0.14 

0.12 

0.09 

0.11 

UAVMaxDetecRange 

0.18 

0.18 

0.13 

0.17 

0.11 

0.18 

0.19 

0.12 

0.16 

TimeBwDetec 

0.02 

0.02 

0.02 

0.03 

0.03 

0.02 

0.02 

0.05 

0.02 

Endurance 

0.23 

0.27 

0.27 

0.19 

0.22 

0.21 

0.25 

0.18 

0.23 

RefuelTime 

0.02 

0.02 

0.03 

0.06 

0.04 

0.04 

0.02 

0.05 

0.03 

SlewRate 

0.09 

0.04 

0.04 

0.11 

0.05 

0.11 

0.06 

0.07 

0.07 

Aperture 

0.27 

0.35 

0.40 

0.22 

0.44 

0.25 

0.31 

0.37 

0.33 


Lawrence Bodin and Saul Gass explain the 10% rule, which refers to the 
consistency level that AHP users have to maintain. It reflects that users were consistent in 
the preference rating in the pairwise matrix. Our preference rating in Table 7 has resulted 
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in 4.1%, which is a satisfactory ratio of consistency (Bodin & Gass 2003). Refer to 
Appendix E for the full calculations. 

b. Design Points Relative Cost Calculation 

Table 9 illustrate the process the author used to calculate the relative cost for each 
design point. First, we break down the cost of one vehicle of the Raven RQ-11 UAV 
based on the calculated UAV capability weights in Table 8. The second row (cost per 
factor) shows the cost of each capability in regards to the relative weights of the factor. 
Next, we compare each Raven RQ-11 capability to the DP capability and compute the 
percentage difference. The resulting percentage is then multiplied by the Raven RQ-11 
capability relative cost (cost per factor row) to calculate the required DP relative cost. 
Table 9 shows (DP 1) relative cost calculation, following this process, has resulted in a 
total relative cost for DP 1 equals $65,440. 

There are some limitations to the relative cost calculation used in this study based 
on the assumptions made by the author. Each design point is an enhancement or a 
decrement from the reference Raven RQ-11 capabilities. The cost for the enhancement is 
proportional and linear to the importance of the characteristic. We alert the readers that 
this is an estimation for the DP cost and not a precise cost value. 


Table 9. Design Point Relative Cost Calculation. 



UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Sum 

$ 34,000.00 

0.050 

0.108 

0.157 

0.024 

0.227 

0.034 

0.073 

0.327 

i 

cost per factor 

$1,693.06 

$3,660.52 

$5,333.03 

$821.63 

$7,729.54 

$1,156.52 

$2,495.72 

$11,109.97 

$34,000.00 

Raven RQ-11 capability 

80 

3000 

1000 

20 

1 

60 

60 

60 


DP capability 

166 

3234 

2078 

10 

2 

53 

299 

89 


% Change 

1.08 

0.08 

1.08 

0.50 

1.00 

0.12 

3.98 

0.48 


% Change cost 

$1,820.04 

$ 285.52 

$5,749.01 

$410.82 

$7,729.54 

$ 134.93 

$9,941.30 

$ 5,369.82 

$31,440.97 


cost 1 UAV 

$65,440.97 


The resulting relative cost for one UAV in Table 9 is multiplied by three (for a set 


of three UAVs) and then added to the cost of the two ground control stations and 
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supporting equipment. A similar methodology is used for all 260 DPs to calculate each 
DP’s total relative cost to the system (see Appendix F). 

c. Low Red Stealth Relative Cost Tradeoff 

All MOEs’ values for low red stealth DPs will be plotted against the total system 
relative cost. This can better illustrate the best DP that maximizes the MOE value for the 
lowest relative cost value in both case scenarios. 

(1) MOE 1 vs. Relative Cost (Low Stealth) 

Figure 23 presents each MOE 1 DP with its total system relative cost value. The 
dashed black line is the MOE #1 threshold. DPs in the red circle represent the DPs that 
achieve the threshold for both MOEs. The orange line represents the relative efficiency 
frontier line that connects the DPs with the highest MOE #1 value at their relative total 
cost value. 


0.6 



0.5 


0.4 1 - 

$200,000.00 


$700,000.00 


$1,200,000.00 $1,700,000.00 

Total cost 


A MOE 1 
• Best MOE 1 


$2,200,000.00 


$2,700,000.00 


Figure 23. Relative Efficiency Frontier for MOE 1 (Low Red Stealth). 
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As shown in Figure 23, DP 79 meets both MOEs’ thresholds with the lowest 
relative total system cost of $935,222. Other DPs meet both MOEs thresholds, but with a 
more expensive relative total system cost. 

The cheapest option that reaches the threshold of MOE 1 only is DP 14 (90.35%). 
It has a relative total system cost that is almost half the DP 79 ($467,611.49), but DP 14’s 
time to detect 40% of the red agents is higher than the MOE 2 threshold of 5.93 hours. 

(2) MOE 2 vs. Relative Cost (Low Stealth) 

Figure 24 presents each MOE 2 DP with its relative system cost. The dashed 
black line represents the MOE 2 threshold of 3.55 hours. Similarly, DPs in the red circle 
represent the DPs that achieve the threshold for both MOEs. The orange line represents 
the relative efficiency frontier line that connects the DPs with the lowest MOE #2 value. 



Figure 24. Relative Efficiency Frontier for MOE 2 (Low Red Stealth). 
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As shown in Figure 24, DP 79 has the lowest relative cost and satisfies both 
MOEs with an MOE 2 value of 3.28 hours. No other DP can reach the MOE 2 threshold 
at cheaper relative cost. Other DPs fall into the relative efficiency frontier curve and 
could deliver less time than DP 79, but they will relatively cost more. For example, DP 
209 can expend the least time to detect 40% of the red agents (1.77 hours) but will have a 
relative cost of exactly double the price $1,870,445.94 as that of DP 79. 

(3) Low Stealth Design Points Relative Cost 

Table 10 presents the low red stealth design points with their relative cost. These 
relative costs serve only as a guide for the GCF because actual UAV component cost 
breakdowns were hard to obtain during the timeframe of this study. Actual costs would 
have to be considered for comparison if GCF needs to purchase a UAV with greater 
capability to fulfill its needs. 
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Table 10. Low Red Stealth DP Relative Cost. 


DP 

Num 

UAV 

UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Red 

Stealth 

MOE2 

time 

To 

Detect 

40% 

(Hr) 

MOE1 

%Red 

Agent 

Detected 

Relative 

Cost 

79 

2 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

3.28 

0.96895 

$ 

935,222.97 

106 

2 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

3.47 

0.95805 

$ 

1,030,993.22 

110 

2 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

3.37 

0.9676 

$ 

998,338.46 

140 

3 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

3.29 

0.91215 

$ 

1,230,764.96 

141 

3 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

3.34 

0.9016 

$ 

1,248,557.99 

144 

3 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

2.42 

0.9175 

$ 

1,402,834.46 

150 

3 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

2.89 

0.9027 

$ 

1,555,820.62 

159 

3 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

2.76 

0.90325 

$ 

1,527,727.19 

163 

3 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

2.67 

0.9112 

$ 

1,548,796.06 

171 

3 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

2.57 

0.9117 

$ 

1,546,489.82 

172 

3 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

3.06 

0.9008 

$ 

1,556,531.10 

175 

3 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

2.53 

0.9127 

$ 

1,497,507.69 

201 

4 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

2.29 

0.9216 

$ 

1,725,460.02 

205 

4 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

2.06 

0.92695 

$ 

1,641,019.94 

206 

4 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

2.34 

0.9341 

$ 

1,664,743.99 

209 

4 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

1.77 

0.9449 

$ 

1,870,445.94 

212 

4 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

2.51 

0.9126 

$ 

1,941,283.65 

215 

4 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

2.08 

0.94435 

$ 

2,074,427.50 

224 

4 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

1.94 

0.942 

$ 

2,036,969.58 

228 

4 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

2.04 

0.93765 

$ 

2,065,061.41 

229 

4 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

2.61 

0.90785 

$ 

1,962,672.06 

233 

4 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

2.54 

0.92015 

$ 

2,078,105.80 

236 

4 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

1.81 

0.94225 

$ 

2,061,986.43 

237 

4 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

2.15 

0.92205 

$ 

2,075,374.80 

240 

4 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

1.84 

0.9488 

$ 

1,996,676.91 


Table 10 shows that DP 79 is the lowest relative cost option in addition to being 
the highest MOE 1 percentage of detection. Therefore, we can conclude that DP 79 will 
provide the best relative cost efficiency. 

d. High Red Stealth Relative Cost Tradeoff 

If GCF decided that it would go with the high red stealth scenario, different 
design point MOE values should be plohed against the relative total system cost. Figure 
21 in the partition tree analysis section shows that seven possible DPs satisfy both MOEs 
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with a high red stealth value; only one of them, though, can achieve both MOEs with less 
than four UAV sets. 

(1) MOE 1 vs. Relative Cost (High Stealth) 

Similarly, Figure 25 represents each MOE 1 DP with its relative total system cost 
for the high red stealth case. DPs in red circles represent the seven DP options that can 
meet both MOEs for the high red stealth scenario. DP 176 meets both MOEs’ thresholds 
with the lowest relative total system cost of $1,577,786 for the high red stealth scenario. 
DP 176 provides a 90.2% detection of the red agents, while DP 241 provides the highest 
possible percentage of red agent detection for the high red stealth DP option of 93.24%. 
This design point option requires an additional $525k, however, when compared to DP 
176. 



Figure 25. Relative Efficiency Frontier for MOE 1 (High Red Stealth). 
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(2) MOE 2 vs. Relative Cost (High Stealth) 

Figure 26 represents the MOE 2 DP for high red stealth with a relative total 
system cost value. DP 176 has the least relative cost and satisfies both MOEs with an 
MOE 2 value of 2.69 hours. Nonetheless, DP 241 can satisfy both MOEs and provide a 
better MOE 2 value of 1.99 hours. 


MOE 2 High Red Stealth DP's 

18 

A 

A 

16 



Total cost 

Figure 26. Relative Efficiency Frontier for MOE 2 (High Red Stealth). 

(3) High Stealth Design Points Relative Cost 

Table 11 represents all design points for the high red stealth scenario and its 
relative cost. DP 176 is the low relative cost option if the GCF decided to account for 
highly stealthy enemies. However, other design point options, such as DP 241 provide a 
higher percentage of detection and less time to detect 40% of the red agents when red 
stealth is high. Although it is relatively more expensive than DP 176, the added benefits 
of DP 241 may justify the cost. 
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Table 11. High Red Stealth DP Options Relative Cost. 


DP 

Num 

UAV 

UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Red 

Stealth 

MOE2 

time 

To 

Detect 

40% 

(Hr) 

MOE1 

%Red 

Agent 

Detected 

Relative 

Cost 

176 

3 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

2.69 

0.9028 

$ 1 , 577 , 786.87 

211 

4 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

2.36 

0.9207 

$ 2 , 049 , 055.26 

221 

4 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

2.58 

0.9004 

$ 1 , 925 , 269.63 

226 

4 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

2.68 

0.9072 

$ 2 , 146 , 678.15 

227 

4 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

2.53 

0.91465 

$ 1 , 867 , 307.31 

241 

4 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

1.99 

0.9324 

$ 2 , 103 , 715.83 

246 

4 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 

2.45 

0.9093 

$ 1 , 755 , 737.39 


e. Design Points Mission Success Probability 

MOE 1 and 2 data for each design point resulted from averaging 40 runs of this 
DP, as discussed in the section on the number of replications. For each design point that 
was successful in meeting the threshold of both MOEs, we ran the replication an 
additional 100 times. This allows calculating the proportion of those runs that effectively 
meet both MOEs’ mission success (see Appendix G). 

We calculated a 95% confidence interval of the proportion over the 100 results for 
each DP. The lower Cl value defines the conservative estimate of a DP’s probability of 
mission success. 

In Figures 27 and 28, we plotted both scenarios’ lower level of the 95% Cl for the 
100 runs. Figure 27 shows that the low red stealth DP 79 has a 97.04% probability of 
mission success. It also shows that there are multiple DP options that achieved a 100% 
probability of mission success. The cheapest DP for the low red stealth scenario that 
scores 100% is DP 205. Although it relatively costs $705,000 USD more than DP 79, it 
increases mission success by only 3%. 
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Figure 27. Low Red Stealth Mission Success. 


The observation regarding the high red stealth DPs in Figure 28 shows that the 
most cost-effective option for high red stealth, DP 176, has a low probability of mission 
success, at only 44.18%. This indicates that if this option is chosen by the GCF, it has a 
low probability of delivering the required MOE thresholds. The other DP option for the 
high red stealth scenario is DP 241; this DP option will provide 100% probability of 
mission success. 
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High Red Mission Success 



Figure 28. High Red Stealth Mission Success. 
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V. CONCLUSION 


This research examines how the addition of UAVs in the detection operation 
along the border of Yemen and Saudi Arabia can help improve the low detection rates of 
infiltrators. Findings from this study provide useful information for decision makers 
seeking suitable acquisition options for an appropriate UAV to support GCF ground 
forces in the detection operation. 

To provide support for the GCF on how to defend their operational areas against 
enemy incursions and reconnaissance, we examined the role of UAV technology for the 
Decisive Storom operation sin Yemen. Using a systems engineering mindset and 
applying modeling and simulation, aided in providing insights to the detection 
capabilities that Gulf Coalition Force’s desires. The experimental design helped 
determine the optimal technical characteristics for a UAV that can enhance the ISR role 
in the operation area. Regression and tree analysis tools were used for exploring and 
examining all design points in this study. Finally, the author conducted a comparative 
cost analysis to identify the desirable option GCF wish to pursue for both cases in this 
study. These outcomes can help the GCF decision makers with their acquisition of UAVs 
for the Decisive Storm operations in Yemen. 

A. PRIMARY FINDINGS 

• The current detection capability of the GCF is insufficient in terms of 
detection frequency and time of detection. This situation drives the GCF’s 
need for enhanced detection methods to protect the border more 
efficiently. 

• Introducing UAVs to support the GCF operation can boost the percentage 
of enemy detections that the GCF can achieve. Despite the number of 
UAV sets employed in the simulations described in this research, the 
addition of this technology made a 49% increase in the percentage of 
enemy detections over the baseline model configuration, which employed 
no UAVs. The time to detect infiltrators also showed promising results 
with the inclusion of UAVs in the detection operation simulations. 

• The introduction of UAVs into the detection operation improved the 
enemy detection rate; however, the UAVs failed to satisfy the operational 
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thresholds of both MOEs because the vehicles did not have the right 
capabilities. 

• To minimize financial obligations and maximize cost efficiency, it is 
essential that decision makers select the right UAV group for the 
operation, as well as the optimal quantity of UAV sets required for the 
mission. 

• Higher capability UAVs will provide the required operational goals and 
might even exceed them, but this option can lead to a substantial financial 
investment and surplus flight performance and sensor capabilities. 

• The relative cost analysis section of this research reveals the desirable 
design point option for both case scenarios. Both design points for high 
and low red stealth scenarios have flight performance capabilities that fall 
into the Group 3 UAV category as described by the U.S. Anny UAS Road 
Map 2010-2035 (Army UAS CoE 2010). 

• The referenced UAV for this study was from the Group 1 UAV category 
as described by the U.S. Anny UAS Road Map 2010-2035 (Anny UAS 
CoE 2010); however, the best design points show a need to procure a 
Group 3 UAV to perform the detection operation on the Saudi-Yemeni 
border. 

B. OTHER IMPORTANT FINDINGS 

• A number of possible DPs can meet the threshold levels for both MOEs in 
either red stealth capability environment. 

• The regression analysis discovered the factors that have the most 
significant effect on the MOEs. For the percentage of red agents detected, 
it specified that the red stealth level, number of UAVs, UAV maximum 
detection range, aperture angle, and the UAV endurance all have an effect 
on MOE 1. Additionally, the red stealth level, number of UAVs, UAV 
maximum detection range, aperture angle, UAV endurance, and UAV 
speed have an effect on the time it takes to detect 40% of the red agents. 
The regression analysis also shows that the red stealth level is a major 
factor in the detection of red agents. 

• The partition tree analysis helped to identify the most suitable UAV 
factors for achieving the operational goals of the GCF in both extreme 
cases (high and low red stealth environment). Both cases have been 
broken down and the number of UAVs is a major factor in detection for 
both cases. Few design points showed that fewer than four UAV sets are 
able to meet both MOEs operational goals. 
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• The partition tree analysis showed that solutions using more UAVs tend to 
give better results or more options to achieve the MOEs’ threshold, but the 
marginal improvement after the third UAV set does not appear to justify 
the cost in the low red stealth case. 

• The relative cost analysis discovered the desirable UAV factor 
combination for both case scenarios. Specifically, DP 79 was the desired 
option for the low red stealth scenario. It has the highest percentage of red 
agent detection and it has a high probability of mission success. 

• The relative cost analysis also discovered the desirable UAV factor 
combination for the high red stealth scenario. Specifically, DP 241 was the 
best design point option with the highest percentage of red agent detection 
and lowest time to detect 40% of the red agents. It also had a 100% 
probability of mission success. The relative cost analysis proved helpful in 
identifying the DPs for the GCF in their detection mission. 

• The best UAV factors combination for the low red stealth scenario is as 
follows: speed of 149 knots, 4641 meters in altitude, maximum detection 
range of 3344 meters, 19 second time between detections, endurance of 4 
hours, 37 minutes of refuel time, slew rate of 201 degrees, and an aperture 
of 116 degrees. 

• The best UAV factors combination for the high red stealth scenario is as 
follows: speed of 106 knots, 5266 meters in altitude, maximum detection 
range of 2734 meters, 7 second time between detections, endurance of 6 
hours, 31 minutes of refuel time, slew rate of 346 degrees, and an aperture 
of 120 degrees. 


C. FUTURE STUDIES 

As mentioned earlier in this thesis, our analyses identified other issues such as 
route development and UAV sensor capabilities that merit more consideration but fell 
outside the scope of this research. The following are potential areas to explore in future 
studies: 

• Use the exact same type and number of GCF detection equipment and 
their technical parameters for future simulations. 

• Model the impact of the friendly forces in the operational area or civilians 
by adding classification factors into the UAV. 

• Study the effect of false targets detections that sensors generate. 
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• Consider a detection method that red agents can use against UAVs and 
allow them to hide before they are detected or cause damage to UAV. 

• Consider adding extra manned aircraft, such as F-16s, in the operational 
area to support detection missions, to simulate the communication and 
data exchange between UAVs and manned aircrafts. 

• Apply data-link communication timings to simulate real life scenario 
between UAVs and the command centers. 

• Allow UAV to change flying paths based on target locations. 

• Use detailed data for the reference UAV for the cost analysis. 
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APPENDIX A. 65 DESIGN POINTS 



UAVSpeed 

UAVAItitude 

UAVMaxDetRng 

UAVMaxTimeBtwDet 

UAVEndurance 

UAVRefuelTime 

UAVSIewRate 

UAVAperture 

Red Stealth 

1 

166 

3234 

2078 

10 

2 

53 

299 

89 

84 

2 

194 

6594 

1328 

11 

3 

38 

224 

105 

79 

3 

187 

4797 

3859 

8 

3 

56 

107 

87 

73 

4 

157 

7453 

3156 

12 

1 

43 

140 

76 

68 

5 

191 

5344 

1563 

5 

2 

35 

135 

97 

77 

6 

144 

7609 

1703 

12 

2 

50 

102 

112 

83 

7 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

8 

178 

6828 

3766 

10 

4 

58 

271 

61 

70 

9 

163 

3156 

1047 

17 

4 

48 

177 

66 

81 

10 

196 

6438 

2453 

16 

1 

38 

337 

83 

78 

11 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

12 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

13 

146 

4094 

1984 

15 

4 

45 

116 

77 

84 

14 

174 

5734 

2266 

18 

3 

59 

74 

60 

75 

15 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

16 

161 

5891 

2641 

16 

2 

59 

360 

94 

74 

17 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

18 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

19 

168 

4953 

2688 

9 

6 

57 

69 

84 

76 

20 

153 

6672 

3578 

6 

5 

35 

191 

70 

81 

21 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

22 

176 

7141 

1000 

7 

5 

51 

205 

113 

68 

23 

200 

5031 

3391 

9 

8 

34 

276 

68 

76 

24 

151 

8000 

3484 

11 

6 

46 

233 

73 

85 

25 

155 

4250 

1281 

14 

7 

52 

341 

69 

67 

26 

170 

6125 

1469 

20 

6 

36 

252 

82 

73 

27 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

28 

189 

7219 

2875 

17 

7 

39 

98 

99 

80 

29 

193 

4484 

1375 

14 

5 

36 

182 

65 

69 

30 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

31 

159 

3625 

3109 

13 

8 

42 

290 

106 

83 

32 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

33 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

34 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 


63 

















































35 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

36 

93 

6203 

1141 

17 

6 

34 

313 

93 

77 

37 

123 

3547 

1844 

13 

8 

47 

280 

104 

83 

38 

89 

5656 

3438 

20 

7 

55 

285 

83 

73 

39 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

40 

108 

7063 

2406 

20 

6 

50 

93 

95 

82 

41 

103 

4172 

1234 

15 

5 

32 

149 

119 

80 

42 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

43 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

44 

138 

7922 

1094 

12 

7 

47 

266 

79 

85 

45 

82 

5422 

1938 

6 

7 

58 

248 

71 

79 

46 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

47 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

48 

131 

6359 

1656 

6 

5 

53 

219 

64 

82 

49 

119 

5109 

2359 

9 

7 

31 

60 

86 

76 

50 

97 

5813 

2828 

18 

3 

30 

126 

62 

73 

51 

133 

3703 

3203 

15 

4 

46 

112 

72 

78 

52 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 

53 

127 

4328 

1422 

19 

4 

55 

229 

110 

69 

54 

101 

6984 

2781 

19 

1 

49 

355 

78 

79 

55 

104 

3859 

4000 

18 

4 

39 

215 

67 

82 

56 

80 

5969 

1609 

16 

1 

56 

144 

113 

74 

57 

129 

3000 

1516 

14 

3 

44 

187 

107 

65 

58 

125 

6750 

3719 

11 

2 

38 

79 

111 

83 

59 

110 

4875 

3531 

5 

3 

54 

168 

98 

78 

60 

99 

7531 

1750 

8 

4 

33 

262 

63 

70 

61 

91 

3781 

2125 

8 

2 

51 

323 

81 

70 

62 

88 

6516 

3625 

11 

4 

54 

238 

115 

81 

63 

116 

3313 

2969 

9 

2 

30 

257 

88 

84 

64 

121 

7375 

1891 

12 

1 

48 

130 

74 

67 

65 

95 

4719 

1188 

7 

3 

41 

88 

80 

72 


See text for data collection methods and details. 
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APPENDIX B. REQUIRED RUN CALCULATION 


A. MOE1 


Calculate sample size required for desired Cl precision 


alpha 

0.05 

Type l error rate 

confidence 

0.95 

= 1-alpha 

z-sub-alpha-over-2 

1.959963985 

(assuming 2-tailed Cl) 

s (sample std dev) 

8.769529432 



Desired Width of Cl (twice the Margin of Error) 

6 


required sample size (n) 

33 


Sample Target 

Difference Size Power Actual Power 

6 8 0.47 0.490119 


Power and Sample Size 

1-Sample Z Test 

Testing mean = null (versus not = null) 
Calculating power for mean = null + difference 
Alpha = 0.05 Assumed standard deviation =8.77 


Difference 

6 


Sample Target 

Size Power Actual Power 

33 0.975 0.975591 
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MOE 2 


Method 2: Calculate sample size required for desired Cl precision 


alpha 

0.05 

Type l error rate 

confidence 

0.95 

= 1-alpha 

z-sub-alpha-over-2 

1.959963985 

(assuming 2-tailed Cl) 

s (sample std dev) 

291 



Desired Width of Cl (twice the Margin of Error) 

180 


required sample size (n) 

41 


Power and Sample Size 

1-Sample Z Test 

Testing mean = null (versus not = null) 
Calculating power for mean = null + difference 
Alpha =0.05 Assumed standard deviation = 291 


Sample Target 

Difference Size Power Actual Power 

180 41 0.975 0.977289 



See text for data collection methods and details. 
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APPENDIX C. BLUE RADAR DESIGN POINTS AND DETECTION 

PROBABILITIES 


DP 

Num 

Radar 

Range 

Tme b/w 
Detection 

Std Dev 

(Alleg2Cas.Red.) 

Mean 

(PercentageRedDetected) 

1 

7 

7313 

60 

8.625543461 

0.4498 

2 

8 

6875 

67 

7.571496821 

0.41885 

3 

7 

7750 

81 

9.532367263 

0.41365 

4 

7 

6500 

69 

8.320348859 

0.4089 

5 

8 

5875 

71 

9.23691035 

0.3795 

6 

8 

7375 

98 

8.80617499 

0.3774 

7 

8 

6750 

89 

9.884947116 

0.37415 

8 

7 

6188 

85 

8.621343521 

0.36665 

9 

7 

7563 

103 

8.689479345 

0.36135 

10 

7 

6313 

93 

8.859443752 

0.3537 

11 

7 

5750 

83 

8.208813275 

0.348 

12 

8 

5188 

68 

7.361542481 

0.3375 

13 

8 

6063 

101 

8.230493645 

0.3349 

14 

7 

7063 

115 

9.272201354 

0.33405 

15 

8 

6625 

116 

8.738215508 

0.3301 

16 

7 

5000 

66 

9.185237599 

0.32825 

17 

7 

5563 

94 

7.445141248 

0.31965 

18 

8 

5625 

108 

8.619447302 

0.3035 

19 

8 

5438 

117 

8.556898552 

0.2872 

20 

7 

4813 

84 

8.318615208 

0.28615 

21 

7 

4125 

61 

9.106070109 

0.2849 

22 

7 

4750 

92 

9.448884075 

0.28095 

23 

7 

4188 

82 

9.508397099 

0.25945 

24 

7 

4500 

102 

8.476059801 

0.2471 

25 

7 

4375 

105 

8.229052661 

0.23995 

26 

6 

7938 

62 

6.842364264 

0.2331 

27 

6 

7688 

74 

6.93814244 

0.20925 

28 

5 

7625 

75 

6.944422222 

0.20665 

29 

6 

7438 

76 

6.535112264 

0.2052 

30 

5 

7500 

78 

7.397851386 

0.2026 

31 

6 

6688 

73 

6.60414594 

0.19755 

32 

5 

7813 

98 

7.08008221 

0.18805 

33 

5 

7250 

88 

7.727365624 

0.18585 

34 

5 

5375 

64 

6.455170855 

0.1857 

35 

6 

8000 

110 

7.259829764 

0.1825 

36 

5 

7188 

96 

6.208141883 

0.1787 

37 

5 

6438 

86 

7.298752967 

0.1768 

38 

6 

6000 

90 

5.917759018 

0.17365 
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39 

6 

7125 

100 

6.341560109 

0.1736 

40 

6 

5063 

68 

6.795737729 

0.1717 

41 

5 

7875 

119 

5.648689433 

0.1714 

42 

5 

5688 

87 

4.790749634 

0.1713 

43 

5 

4938 

65 

4.635357425 

0.17095 

44 

5 

6250 

97 

6.299725269 

0.16835 

45 

6 

6938 

112 

5.461050899 

0.1647 

46 

5 

5813 

95 

6.002349967 

0.1633 

47 

6 

7000 

114 

5.457939406 

0.16085 

48 

6 

4875 

80 

6.29183435 

0.1601 

49 

4 

6563 

63 

4.228641656 

0.15725 

50 

5 

4438 

77 

5.679246971 

0.1549 

51 

6 

4000 

70 

6.13418333 

0.1525 

52 

4 

6375 

72 

5.355981559 

0.15135 

53 

5 

5500 

111 

6.339487563 

0.15125 

54 

6 

5313 

107 

6.129740447 

0.14875 

55 

6 

4563 

104 

5.775223383 

0.14135 

56 

6 

4313 

106 

4.955636522 

0.13715 

57 

5 

4250 

99 

4.965973967 

0.13685 

58 

4 

5938 

79 

4.711143364 

0.1368 

59 

5 

4688 

120 

5.496852246 

0.1304 

60 

4 

6813 

113 

5.323147324 

0.1277 

61 

6 

4063 

118 

5.005381719 

0.1253 

62 

4 

6125 

109 

4.933142755 

0.1237 

63 

4 

5250 

91 

4.091814844 

0.12155 

64 

4 

4625 

83 

4.327535569 

0.11425 

65 

4 

5125 

113 

4.291837811 

0.10825 


See text for data collection methods and details. 
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APPENDIX D. DETAILED PARTITION TREE ANALYSIS 


A. PARTITION TREE ANALYSIS FOR DESIGN POINTS ACHIEVED 

BOTH MOE’S 

The partition tree in Figure 29 shows up to three splits of each MOE’s options 
using Red Stealth as the first split factor. The Red Stealth factor is a non-controllable 
factor, and it depends on how well the red agents hide from detection. Splitting the tree 
according to the Red Stealth factor as a first split will assist the GCF in determining the 
ideal UAV in both extreme cases for detecting red agents at their most detectable (best 
case) and least detectable (worst case). We will further split the tree to see how the UAV 
factors contribute to success in the highest and lowest levels of detectability cases. 

The top box in Figure 29 shows a count of 32 DPs in the Red/Blue bar out of a 
total of 260 DPs. This means that 32 DPs accomplished the requirement for both MOEs’ 
threshold by detecting 90% of the red agents within 3.55 hours. Blue means “YES” and 
red means “NO.” 



Figure 29. Partition Tree for DPs Achieved Both MOEs. 
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Splitting the 260 design points using the Red Stealth factor resulted in 96 DPs 
(top left box) having a red stealth value of less than 73 (Red Stealth<73) among all 260 
DPs. Twenty-five of those DPs only can achieve the detection of 90% of the red agents in 
less than 3.55 hours, while 71 of the DPs failed. On the other side (top right box), 164 
DPs (top right box) have a red stealth value of higher than or equal to 73 (Red 
Stealth>=73) out of 260 DPs. Only seven DPs can achieve the detection of 90% of the 
red agents in less than or equal to 3.55 hours, while 157 could not. 

Further analysis will be broken down based on best and worst case red agent 
concealment scenarios. 

B. LOW RED STEALTH SCENARIO 

The second split after the Red Stealth factor is how many sets of UAVs can 
satisfy the required criteria for both MOEs. The left side box in Figure 30 represents a 
possible 48 DP options among the 260 DPs that have three sets or more than three sets of 
UAVs. There are 22 DPs in that box that can achieve both MOEs. A third split in same 
figure under the Red Stealth factor is when the UAV sensor can detect a maximum range 
of more than or equal to 1797 meters; 22 DPs out of 32 DPs are possible options. Finally, 
a fourth split is to check whether the aperture angle width is greater than or equal to 91 
degrees. Nineteen DPs are in the blue area, which means 19 DPs can achieve the best 
case MOE 1 and 2 thresholds if the UAV aperture angle width is greater than or equal to 
91 degrees and the sensor maximum detection range is greater than 1797 meters. 
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Figure 30. Low Red Stealth Partition Tree. 


There is another path to success with fewer UAV sets if the GCF decided to go 
for a low number of UAVs solution. Looking at the right side portion of the second split 
will detennine what capabilities are required to achieve the best MOE 1 and 2 criteria. 
The right side shows that with fewer than three UAVs, there are three DPs than can 
achieve the best MOE 1 and 2, while 45 out of the 48 DPs could not. 

If the fewer than three UAV sets path was selected, a third split shows three DPs 
out of 24 in the blue area, which means that there should be at least two or more UAV 
sets to satisfy the best case MOE’s criteria; fewer than two UAVs is not a possible 
option. Further splitting shows that UAV aperture is required to be greater than or equal 
to 103 degrees for fewer than two UAVs to satisfy the best case MOEs. Three DPs are in 
the blue area if the aperture is greater than 103 degrees. 
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C. HIGH RED STEALTH SCENARIO 

Figure 31 shows having a red stealth value of at least 73 resulted in 164 DPs 
among all 260 DPs. Only seven DPs met the criteria of being capable of detecting 90% of 
the red agents in 3.55 hours or less. This shows the higher the red stealth value, the harder 
it is to detect red agents or the fewer options we have to be able to detect them. 

The second split is how many UAV sets can satisfy the required criteria for both 
MOEs. The box on the left side presents a possible 41 DP options among the 260 DPs 
that have at least four sets of UAVs. There are six DPs in that box in blue, which means 
that those DPs are possible options to achieve both MOEs’ criteria. 

A third split under the Red Stealth factor is to check whether the aperture angle 
width is at least 90 degrees; the same six DPs from the earlier split are the only options. 
Anything with less than a 90-degree aperture is not an option. Finally, a fourth split 
occurs when the UAV sensor can detect a maximum range of at least 2172 meters. Again, 
the same six DPs fall into this UAV capability and anything less than 2172 meters sensor 
detection is not an option. This means the MOE 1 and MOE 2 threshold is achievable 
when red stealth is high if at least four UAV sets are used, sensor maximum detection 
range is at least 2172 meters, and the aperture angle width is at least 90 degrees. 
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Figure 31. High Red Stealth Partition Tree 


If there is a requirement to reduce the number of UAV sets, while using high 
value for red stealth, only one possible DP option can achieve the MOE 1 and MOE 2 
threshold. The box on the right shows a total of 123 DPs available for fewer than four 
UAVs among the 260 DPs; only one is in the blue area. Further splitting this box to 
explore what is the single option that can satisfy the requirement yields a sensor 
capability having an aperture of at least 119 degrees. Only one DP out of five in this box 
falls into the blue area. 
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APPENDIX E. ANALYTIC HIERARCHY PROCESS 


A. PAIRWISE COMPARISON AND COLUMN SUM 



UAVSpeed 

UAVAltit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefuelTime 

SlewRate 

Aperture 

UAVSpeed 

1 

1/3 

1/4 

3 

1/5 

2 

1/2 

1/6 

UAVAltit 

3 

1 

1/2 

5 

1/3 

4 

2 

1/4 

UAVMaxDetecRange 

4 

2 

1 

6 

1/2 

5 

3 

1/3 

TimeBwDetec 

1/3 

1/5 

1/6 

1 

1/7 

1/2 

1/4 

1/8 

Endurance 

5 

3 

2 

7 

1 

6 

4 

1/2 

RefuelTime 

1/2 

1/4 

1/5 

2 

1/6 

1 

1/3 

1/7 

SlewRate 

2 

1/2 

1/3 

4 

1/4 

3 

1 

1/5 

Aperture 

6 

4 

3 

8 

2 

7 

5 

1 








column sum 

21.833333 

11.28333 

7.45 


36 

4.5928571 

28.5 

16.08333 

2.717857 


B. NORMALIZATION 



UAVSpeed 

UAVAltit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefulTime 

SlewRate 

Aperture 

UAVSpeed 

0.0458015 

0.029542 

0.033557047 

0.083333333 

0.0435459 

0.07017544 

0.031088 

0.061323 

UAVAltit 

0.1374046 

0.088626 

0.067114094 

0.138888889 

0.0725765 

0.14035088 

0.124352 

0.091984 

UAVMaxDetecRange 

0.1832061 

0.177253 

0.134228188 

0.166666667 

0.1088647 

0.1754386 

0.186528 

0.122646 

TimeBwDetec 

0.0152672 

0.017725 

0.022371365 

0.027777778 

0.0311042 

0.01754386 

0.015544 

0.045992 

Endurance 

0.2290076 

0.265879 

0.268456376 

0.194444444 

0.2177294 

0.21052632 

0.248705 

0.183968 

RefulTime 

0.0229008 

0.022157 

0.026845638 

0.055555556 

0.0362882 

0.03508772 

0.020725 

0.052562 

SlewRate 

0.0916031 

0.044313 

0.044742729 

0.111111111 

0.0544323 

0.10526316 

0.062176 

0.073587 

Aperture 

0.2748092 

0.354505 

0.402684564 

0.222222222 

0.4354588 

0.24561404 

0.310881 

0.367937 


C. SCORES AND CONSISTENCY 



UAVSpee 

UAVAltit 

UAVMaxD 

TimeBwDe 

Endurance 

RefulTime 

SlewRate 

Aperture 

Score 

consistency 

UAVSpeed 

0.0458 

0.029542 

0.033557 

0.0833333 

0.0435459 

0.070175 

0.031088 

0.061323 

0.04979578 

8.074073637 

UAVAltit 

0.1374 

0.088626 

0.067114 

0.1388889 

0.0725765 

0.140351 

0.124352 

0.091984 

0.10766222 

8.328310151 

UAVMaxDetecRange 

0.18321 

0.177253 

0.134228 

0.1666667 

0.1088647 

0.175439 

0.186528 

0.122646 

0.15685387 

8.474347312 

TimeBwDetec 

0.01527 

0.017725 

0.022371 

0.0277778 

0.0311042 

0.017544 

0.015544 

0.045992 

0.02416572 

8.157015878 

Endurance 

0.22901 

0.265879 

0.268456 

0.1944444 

0.2177294 

0.210526 

0.248705 

0.183968 

0.22733952 

8.547836835 

RefulTime 

0.0229 

0.022157 

0.026846 

0.0555556 

0.0362882 

0.035088 

0.020725 

0.052562 

0.03401529 

8.07193913 

SlewRate 

0.0916 

0.044313 

0.044743 

0.1111111 

0.0544323 

0.105263 

0.062176 

0.073587 

0.07340364 

8.174082069 

Aperture 

0.27481 

0.354505 

0.402685 

0.2222222 

0.4354588 

0.245614 

0.310881 

0.367937 

0.32676396 

8.507300303 















Cl 

0.04169474 






CR 

0.02957074 



See text for data collection methods and details. 
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APPENDIX F. TOTAL SYSTEM RELATIVE COST CALCULATION 



1 UAV set Cost 

Total system 
relative cost 

DP 

No# of 

UAVs 

1 UAV cost 

3 UAV cost 

2 GCS + Supp 
Equip 

cost 


0 

1 

$ 65,440.97 

$ 196,322.92 

$ 198,000.00 

$ 394,322.92 

$ 394,322.92 

1 

1 

$ 73,954.14 

$ 221,862.43 

$ 198,000.00 

$419,862.43 

$419,862.43 

2 

1 

$ 76,687.89 

$ 230,063.67 

$ 198,000.00 

$ 428,063.67 

$428,063.67 

3 

1 

$ 59,507.63 

$ 178,522.90 

$ 198,000.00 

$ 376,522.90 

$ 376,522.90 

4 

1 

$ 61,010.15 

$ 183,030.46 

$ 198,000.00 

$ 381,030.46 

$ 381,030.46 

5 

1 

$ 64,353.94 

$ 193,061.82 

$ 198,000.00 

$ 391,061.82 

$ 391,061.82 

6 

1 

$ 77,788.33 

$ 233,365.00 

$ 198,000.00 

$431,365.00 

$431,365.00 

7 

1 

$ 88,095.77 

$ 264,287.31 

$ 198,000.00 

$462,287.31 

$462,287.31 

8 

1 

$ 65,718.38 

$ 197,155.15 

$ 198,000.00 

$ 395,155.15 

$ 395,155.15 

9 

1 

$ 64,767.91 

$ 194,303.73 

$ 198,000.00 

$ 392,303.73 

$ 392,303.73 

10 

1 

$ 70,751.66 

$ 212,254.99 

$ 198,000.00 

$410,254.99 

$410,254.99 

11 

1 

$ 72,728.67 

$ 218,186.00 

$ 198,000.00 

$416,186.00 

$416,186.00 

12 

1 

$ 71,139.68 

$ 213,419.04 

$ 198,000.00 

$411,419.04 

$411,419.04 

13 

1 

$ 62,219.77 

$ 186,659.31 

$ 198,000.00 

$ 384,659.31 

$ 384,659.31 

14 

1 

$ 89,870.50 

$ 269,611.49 

$ 198,000.00 

$467,611.49 

$467,611.49 

15 

1 

$ 74,680.66 

$ 224,041.97 

$ 198,000.00 

$422,041.97 

$422,041.97 

16 

1 

$ 104,754.60 

$ 314,263.81 

$ 198,000.00 

$ 512,263.81 

$ 512,263.81 

17 

1 

$ 95,773.64 

$ 287,320.91 

$ 198,000.00 

$485,320.91 

$485,320.91 

18 

1 

$ 91,223.31 

$ 273,669.92 

$ 198,000.00 

$471,669.92 

$471,669.92 

19 

1 

$ 93,049.80 

$ 279,149.41 

$ 198,000.00 

$477,149.41 

$477,149.41 

20 

1 

$ 106,868.96 

$ 320,606.87 

$ 198,000.00 

$ 518,606.87 

$ 518,606.87 

21 

1 

$ 88,555.26 

$ 265,665.77 

$ 198,000.00 

$463,665.77 

$463,665.77 

22 

1 

$ 117,294.83 

$ 351,884.49 

$ 198,000.00 

$ 549,884.49 

$ 549,884.49 

23 

1 

$ 103,741.17 

$ 311,223.51 

$ 198,000.00 

$ 509,223.51 

$ 509,223.51 

24 

1 

$ 98,743.79 

$ 296,231.38 

$ 198,000.00 

$494,231.38 

$494,231.38 

25 

1 

$ 93,389.22 

$ 280,167.65 

$ 198,000.00 

$478,167.65 

$478,167.65 

26 

1 

$ 94,439.14 

$ 283,317.41 

$ 198,000.00 

$481,317.41 

$481,317.41 

27 

1 

$ 107,161.53 

$321,484.58 

$ 198,000.00 

$ 519,484.58 

$ 519,484.58 

28 

1 

$ 77,829.81 

$ 233,489.42 

$ 198,000.00 

$431,489.42 

$ 431,489.42 

29 

1 

$ 103,747.47 

$ 311,242.40 

$ 198,000.00 

$ 509,242.40 

$ 509,242.40 

30 

1 

$ 120,507.79 

$ 361,523.36 

$ 198,000.00 

$ 559,523.36 

$ 559,523.36 

31 

1 

$ 112,889.85 

$338,669.54 

$ 198,000.00 

$ 536,669.54 

$ 536,669.54 

32 

1 

$ 89,608.94 

$ 268,826.83 

$ 198,000.00 

$466,826.83 

$466,826.83 

33 

1 

$ 106,088.45 

$ 318,265.35 

$ 198,000.00 

$ 516,265.35 

$ 516,265.35 
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34 

1 

$ 97 , 556.00 

$ 292 , 668.01 

$ 198 , 000.00 

$ 490 , 668.01 

$ 490 , 668.01 

35 

1 

$ 94 , 841.53 

$ 284 , 524.60 

$ 198 , 000.00 

$ 482 , 524.60 

$ 482 , 524.60 

36 

1 

$ 112 , 021.79 

$ 336 , 065.37 

$ 198 , 000.00 

$ 534 , 065.37 

$ 534 , 065.37 

37 

1 

$ 110 , 524.60 

$ 331 , 573.81 

$ 198 , 000.00 

$ 529 , 573.81 

$ 529 , 573.81 

38 

1 

$ 107 , 175.48 

$ 321 , 526.45 

$ 198 , 000.00 

$ 519 , 526.45 

$ 519 , 526.45 

39 

1 

$ 93 , 742.31 

$ 281 , 226.93 

$ 198 , 000.00 

$ 479 , 226.93 

$ 479 , 226.93 

40 

1 

$ 83 , 454.82 

$ 250 , 364.45 

$ 198 , 000.00 

$ 448 , 364.45 

$ 448 , 364.45 

41 

1 

$ 105 , 832.20 

$ 317 , 496.61 

$ 198 , 000.00 

$ 515 , 496.61 

$ 515 , 496.61 

42 

1 

$ 106 , 947.90 

$ 320 , 843.70 

$ 198 , 000.00 

$ 518 , 843.70 

$ 518 , 843.70 

43 

1 

$ 100 , 777.76 

$ 302 , 333.29 

$ 198 , 000.00 

$ 500 , 333.29 

$ 500 , 333.29 

44 

1 

$ 98 , 847.69 

$ 296 , 543.06 

$ 198 , 000.00 

$ 494 , 543.06 

$ 494 , 543.06 

45 

1 

$ 100 , 389.74 

$ 301 , 169.23 

$ 198 , 000.00 

$ 499 , 169.23 

$ 499 , 169.23 

46 

1 

$ 109 , 309.65 

$ 327 , 928.96 

$ 198 , 000.00 

$ 525 , 928.96 

$ 525 , 928.96 

47 

1 

$ 81 , 658.93 

$ 244 , 976.79 

$ 198 , 000.00 

$ 442 , 976.79 

$ 442 , 976.79 

48 

1 

$ 96 , 848.77 

$ 290 , 546.30 

$ 198 , 000.00 

$ 488 , 546.30 

$ 488 , 546.30 

49 

1 

$ 66 , 776.04 

$ 200 , 328.12 

$ 198 , 000.00 

$ 398 , 328.12 

$ 398 , 328.12 

50 

1 

$ 75 , 776.95 

$ 227 , 330.85 

$ 198 , 000.00 

$ 425 , 330.85 

$ 425 , 330.85 

51 

1 

$ 80 , 311.45 

$ 240 , 934.35 

$ 198 , 000.00 

$ 438 , 934.35 

$ 438 , 934.35 

52 

1 

$ 78 , 479.62 

$ 235 , 438.86 

$ 198 , 000.00 

$ 433 , 438.86 

$ 433 , 438.86 

53 

1 

$ 64 , 660.47 

$ 193 , 981.40 

$ 198 , 000.00 

$ 391 , 981.40 

$ 391 , 981.40 

54 

1 

$ 82 , 974.17 

$ 248 , 922.50 

$ 198 , 000.00 

$ 446 , 922.50 

$ 446 , 922.50 

55 

1 

$ 54 , 419.76 

$ 163 , 259.28 

$ 198 , 000.00 

$ 361 , 259.28 

$ 361 , 259.28 

56 

1 

$ 67 , 788.25 

$ 203 , 364.76 

$ 198 , 000.00 

$ 401 , 364.76 

$ 401 , 364.76 

57 

1 

$ 72 , 785.63 

$ 218 , 356.89 

$ 198 , 000.00 

$ 416 , 356.89 

$ 416 , 356.89 

58 

1 

$ 78 , 140.21 

$ 234 , 420.62 

$ 198 , 000.00 

$ 432 , 420.62 

$ 432 , 420.62 

59 

1 

$ 77 , 090.29 

$ 231 , 270.86 

$ 198 , 000.00 

$ 429 , 270.86 

$ 429 , 270.86 

60 

1 

$ 64 , 409.49 

$ 193 , 228.48 

$ 198 , 000.00 

$ 391 , 228.48 

$ 391 , 228.48 

61 

1 

$ 93 , 720.78 

$ 281 , 162.34 

$ 198 , 000.00 

$ 479 , 162.34 

$ 479 , 162.34 

62 

1 

$ 67 , 783.18 

$ 203 , 349.54 

$ 198 , 000.00 

$ 401 , 349.54 

$ 401 , 349.54 

63 

1 

$ 51 , 021.64 

$ 153 , 064.91 

$ 198 , 000.00 

$ 351 , 064.91 

$ 351 , 064.91 

64 

1 

$ 58 , 644.91 

$ 175 , 934.73 

$ 198 , 000.00 

$ 373 , 934.73 

$ 373 , 934.73 

65 

2 

$ 65 , 440.97 

$ 196 , 322.92 

$ 198 , 000.00 

$ 394 , 322.92 

$ 788 , 645.84 

66 

2 

$ 73 , 954.14 

$ 221 , 862.43 

$ 198 , 000.00 

$ 419 , 862.43 

$ 839 , 724.86 

67 

2 

$ 76 , 687.89 

$ 230 , 063.67 

$ 198 , 000.00 

$ 428 , 063.67 

$ 856 , 127.35 

68 

2 

$ 59 , 507.63 

$ 178 , 522.90 

$ 198 , 000.00 

$ 376 , 522.90 

$ 753 , 045.80 

69 

2 

$ 61 , 010.15 

$ 183 , 030.46 

$ 198 , 000.00 

$ 381 , 030.46 

$ 762 , 060.92 

70 

2 

$ 64 , 353.94 

$ 193 , 061.82 

$ 198 , 000.00 

$ 391 , 061.82 

$ 782 , 123.64 

71 

2 

$ 77 , 788.33 

$ 233 , 365.00 

$ 198 , 000.00 

$ 431 , 365.00 

$ 862 , 730.01 

72 

2 

$ 88 , 095.77 

$ 264 , 287.31 

$ 198 , 000.00 

$ 462 , 287.31 

$ 924 , 574.63 

73 

2 

$ 65 , 718.38 

$ 197 , 155.15 

$ 198 , 000.00 

$ 395 , 155.15 

$ 790 , 310.31 
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2 

$ 64 , 767.91 

$ 194 , 303.73 

$ 198 , 000.00 

$ 392 , 303.73 

$ 784 , 607.46 

75 

2 

$ 70 , 751.66 

$ 212 , 254.99 

$ 198 , 000.00 

$ 410 , 254.99 

$ 820 , 509.97 

76 

2 

$ 72 , 728.67 

$ 218 , 186.00 

$ 198 , 000.00 

$ 416 , 186.00 

$ 832 , 371.99 

77 

2 

$ 71 , 139.68 

$ 213 , 419.04 

$ 198 , 000.00 

$ 411 , 419.04 

$ 822 , 838.08 

78 

2 

$ 62 , 219.77 

$ 186 , 659.31 

$ 198 , 000.00 

$ 384 , 659.31 

$ 769 , 318.63 

79 

2 

$ 89 , 870.50 

$ 269 , 611.49 

$ 198 , 000.00 

$ 467 , 611.49 

$ 935 , 222.97 

80 

2 

$ 74 , 680.66 

$ 224 , 041.97 

$ 198 , 000.00 

$ 422 , 041.97 

$ 844 , 083.95 

81 

2 

$ 104 , 754.60 

$ 314 , 263.81 

$ 198 , 000.00 

$ 512 , 263.81 

$ 1 , 024 , 527.63 

82 

2 

$ 95 , 773.64 

$ 287 , 320.91 

$ 198 , 000.00 

$ 485 , 320.91 

$ 970 , 641.83 

83 

2 

$ 91 , 223.31 

$ 273 , 669.92 

$ 198 , 000.00 

$ 471 , 669.92 

$ 943 , 339.84 

84 

2 

$ 93 , 049.80 

$ 279 , 149.41 

$ 198 , 000.00 

$ 477 , 149.41 

$ 954 , 298.81 

85 

2 

$ 106 , 868.96 

$ 320 , 606.87 

$ 198 , 000.00 

$ 518 , 606.87 

$ 1 , 037 , 213.75 

86 

2 

$ 88 , 555.26 

$ 265 , 665.77 

$ 198 , 000.00 

$ 463 , 665.77 

$ 927 , 331.55 

87 

2 

$ 117 , 294.83 

$ 351 , 884.49 

$ 198 , 000.00 

$ 549 , 884.49 

$ 1 , 099 , 768.99 

88 

2 

$ 103 , 741.17 

$ 311 , 223.51 

$ 198 , 000.00 

$ 509 , 223.51 

$ 1 , 018 , 447.02 

89 

2 

$ 98 , 743.79 

$ 296 , 231.38 

$ 198 , 000.00 

$ 494 , 231.38 

$ 988 , 462.77 

90 

2 

$ 93 , 389.22 

$ 280 , 167.65 

$ 198 , 000.00 

$ 478 , 167.65 

$ 956 , 335.30 

91 

2 

$ 94 , 439.14 

$ 283 , 317.41 

$ 198 , 000.00 

$ 481 , 317.41 

$ 962 , 634.82 

92 

2 

$ 107 , 161.53 

$ 321 , 484.58 

$ 198 , 000.00 

$ 519 , 484.58 

$ 1 , 038 , 969.16 

93 

2 

$ 77 , 829.81 

$ 233 , 489.42 

$ 198 , 000.00 

$ 431 , 489.42 

$ 862 , 978.84 

94 

2 

$ 103 , 747.47 

$ 311 , 242.40 

$ 198 , 000.00 

$ 509 , 242.40 

$ 1 , 018 , 484.79 

95 

2 

$ 120 , 507.79 

$ 361 , 523.36 

$ 198 , 000.00 

$ 559 , 523.36 

$ 1 , 119 , 046.72 

96 

2 

$ 112 , 889.85 

$ 338 , 669.54 

$ 198 , 000.00 

$ 536 , 669.54 

$ 1 , 073 , 339.08 

97 

2 

$ 89 , 608.94 

$ 268 , 826.83 

$ 198 , 000.00 

$ 466 , 826.83 

$ 933 , 653.66 

98 

2 

$ 106 , 088.45 

$ 318 , 265.35 

$ 198 , 000.00 

$ 516 , 265.35 

$ 1 , 032 , 530.71 

99 

2 

$ 97 , 556.00 

$ 292 , 668.01 

$ 198 , 000.00 

$ 490 , 668.01 

$ 981 , 336.03 

100 

2 

$ 94 , 841.53 

$ 284 , 524.60 

$ 198 , 000.00 

$ 482 , 524.60 

$ 965 , 049.20 

101 

2 

$ 112 , 021.79 

$ 336 , 065.37 

$ 198 , 000.00 

$ 534 , 065.37 

$ 1 , 068 , 130.74 

102 

2 

$ 110 , 524.60 

$ 331 , 573.81 

$ 198 , 000.00 

$ 529 , 573.81 

$ 1 , 059 , 147.62 

103 

2 

$ 107 , 175.48 

$ 321 , 526.45 

$ 198 , 000.00 

$ 519 , 526.45 

$ 1 , 039 , 052.90 

104 

2 

$ 93 , 742.31 

$ 281 , 226.93 

$ 198 , 000.00 

$ 479 , 226.93 

$ 958 , 453.85 

105 

2 

$ 83 , 454.82 

$ 250 , 364.45 

$ 198 , 000.00 

$ 448 , 364.45 

$ 896 , 728.89 

106 

2 

$ 105 , 832.20 

$ 317 , 496.61 

$ 198 , 000.00 

$ 515 , 496.61 

$ 1 , 030 , 993.22 

107 

2 

$ 106 , 947.90 

$ 320 , 843.70 

$ 198 , 000.00 

$ 518 , 843.70 

$ 1 , 037 , 687.40 

108 

2 

$ 100 , 777.76 

$ 302 , 333.29 

$ 198 , 000.00 

$ 500 , 333.29 

$ 1 , 000 , 666.57 

109 

2 

$ 98 , 847.69 

$ 296 , 543.06 

$ 198 , 000.00 

$ 494 , 543.06 

$ 989 , 086.12 

110 

2 

$ 100 , 389.74 

$ 301 , 169.23 

$ 198 , 000.00 

$ 499 , 169.23 

$ 998 , 338.46 

111 

2 

$ 109 , 309.65 

$ 327 , 928.96 

$ 198 , 000.00 

$ 525 , 928.96 

$ 1 , 051 , 857.92 

112 

2 

$ 81 , 658.93 

$ 244 , 976.79 

$ 198 , 000.00 

$ 442 , 976.79 

$ 885 , 953.57 

113 

2 

$ 96 , 848.77 

$ 290 , 546.30 

$ 198 , 000.00 

$ 488 , 546.30 

$ 977 , 092.59 
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114 

2 

$ 66 , 776.04 

$ 200 , 328.12 

$ 198 , 000.00 

$ 398 , 328.12 

$ 796 , 656.23 

115 

2 

$ 75 , 776.95 

$ 227 , 330.85 

$ 198 , 000.00 

$ 425 , 330.85 

$ 850 , 661.70 

116 

2 

$ 80 , 311.45 

$ 240 , 934.35 

$ 198 , 000.00 

$ 438 , 934.35 

$ 877 , 868.70 

117 

2 

$ 78 , 479.62 

$ 235 , 438.86 

$ 198 , 000.00 

$ 433 , 438.86 

$ 866 , 877.73 

118 

2 

$ 64 , 660.47 

$ 193 , 981.40 

$ 198 , 000.00 

$ 391 , 981.40 

$ 783 , 962.79 

119 

2 

$ 82 , 974.17 

$ 248 , 922.50 

$ 198 , 000.00 

$ 446 , 922.50 

$ 893 , 845.00 

120 

2 

$ 54 , 419.76 

$ 163 , 259.28 

$ 198 , 000.00 

$ 361 , 259.28 

$ 722 , 518.55 

121 

2 

$ 67 , 788.25 

$ 203 , 364.76 

$ 198 , 000.00 

$ 401 , 364.76 

$ 802 , 729.52 

122 

2 

$ 72 , 785.63 

$ 218 , 356.89 

$ 198 , 000.00 

$ 416 , 356.89 

$ 832 , 713.78 

123 

2 

$ 78 , 140.21 

$ 234 , 420.62 

$ 198 , 000.00 

$ 432 , 420.62 

$ 864 , 841.24 

124 

2 

$ 77 , 090.29 

$ 231 , 270.86 

$ 198 , 000.00 

$ 429 , 270.86 

$ 858 , 541.72 

125 

2 

$ 64 , 409.49 

$ 193 , 228.48 

$ 198 , 000.00 

$ 391 , 228.48 

$ 782 , 456.95 

126 

2 

$ 93 , 720.78 

$ 281 , 162.34 

$ 198 , 000.00 

$ 479 , 162.34 

$ 958 , 324.68 

127 

2 

$ 67 , 783.18 

$ 203 , 349.54 

$ 198 , 000.00 

$ 401 , 349.54 

$ 802 , 699.07 

128 

2 

$ 51 , 021.64 

$ 153 , 064.91 

$ 198 , 000.00 

$ 351 , 064.91 

$ 702 , 129.82 

129 

2 

$ 58 , 644.91 

$ 175 , 934.73 

$ 198 , 000.00 

$ 373 , 934.73 

$ 747 , 869.46 

130 

3 

$ 65 , 440.97 

$ 196 , 322.92 

$ 198 , 000.00 

$ 394 , 322.92 

$ 1 , 182 , 968.75 

131 

3 

$ 73 , 954.14 

$ 221 , 862.43 

$ 198 , 000.00 

$ 419 , 862.43 

$ 1 , 259 , 587.29 

132 

3 

$ 76 , 687.89 

$ 230 , 063.67 

$ 198 , 000.00 

$ 428 , 063.67 

$ 1 , 284 , 191.02 

133 

3 

$ 59 , 507.63 

$ 178 , 522.90 

$ 198 , 000.00 

$ 376 , 522.90 

$ 1 , 129 , 568.71 

134 

3 

$ 61 , 010.15 

$ 183 , 030.46 

$ 198 , 000.00 

$ 381 , 030.46 

$ 1 , 143 , 091.39 

135 

3 

$ 64 , 353.94 

$ 193 , 061.82 

$ 198 , 000.00 

$ 391 , 061.82 

$ 1 , 173 , 185.46 

136 

3 

$ 77 , 788.33 

$ 233 , 365.00 

$ 198 , 000.00 

$ 431 , 365.00 

$ 1 , 294 , 095.01 

137 

3 

$ 88 , 095.77 

$ 264 , 287.31 

$ 198 , 000.00 

$ 462 , 287.31 

$ 1 , 386 , 861.94 

138 

3 

$ 65 , 718.38 

$ 197 , 155.15 

$ 198 , 000.00 

$ 395 , 155.15 

$ 1 , 185 , 465.46 

139 

3 

$ 64 , 767.91 

$ 194 , 303.73 

$ 198 , 000.00 

$ 392 , 303.73 

$ 1 , 176 , 911.19 

140 

3 

$ 70 , 751.66 

$ 212 , 254.99 

$ 198 , 000.00 

$ 410 , 254.99 

$ 1 , 230 , 764.96 

141 

3 

$ 72 , 728.67 

$ 218 , 186.00 

$ 198 , 000.00 

$ 416 , 186.00 

$ 1 , 248 , 557.99 

142 

3 

$ 71 , 139.68 

$ 213 , 419.04 

$ 198 , 000.00 

$ 411 , 419.04 

$ 1 , 234 , 257.13 

143 

3 

$ 62 , 219.77 

$ 186 , 659.31 

$ 198 , 000.00 

$ 384 , 659.31 

$ 1 , 153 , 977.94 

144 

3 

$ 89 , 870.50 

$ 269 , 611.49 

$ 198 , 000.00 

$ 467 , 611.49 

$ 1 , 402 , 834.46 

145 

3 

$ 74 , 680.66 

$ 224 , 041.97 

$ 198 , 000.00 

$ 422 , 041.97 

$ 1 , 266 , 125.92 

146 

3 

$ 104 , 754.60 

$ 314 , 263.81 

$ 198 , 000.00 

$ 512 , 263.81 

$ 1 , 536 , 791.44 

147 

3 

$ 95 , 773.64 

$ 287 , 320.91 

$ 198 , 000.00 

$ 485 , 320.91 

$ 1 , 455 , 962.74 

148 

3 

$ 91 , 223.31 

$ 273 , 669.92 

$ 198 , 000.00 

$ 471 , 669.92 

$ 1 , 415 , 009.76 

149 

3 

$ 93 , 049.80 

$ 279 , 149.41 

$ 198 , 000.00 

$ 477 , 149.41 

$ 1 , 431 , 448.22 

150 

3 

$ 106 , 868.96 

$ 320 , 606.87 

$ 198 , 000.00 

$ 518 , 606.87 

$ 1 , 555 , 820.62 

151 

3 

$ 88 , 555.26 

$ 265 , 665.77 

$ 198 , 000.00 

$ 463 , 665.77 

$ 1 , 390 , 997.32 

152 

3 

$ 117 , 294.83 

$ 351 , 884.49 

$ 198 , 000.00 

$ 549 , 884.49 

$ 1 , 649 , 653.48 

153 

3 

$ 103 , 741.17 

$ 311 , 223.51 

$ 198 , 000.00 

$ 509 , 223.51 

$ 1 , 527 , 670.53 
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154 

3 

$ 98 , 743.79 

$ 296 , 231.38 

$ 198 , 000.00 

$ 494 , 231.38 

$ 1 , 482 , 694.15 

155 

3 

$ 93 , 389.22 

$ 280 , 167.65 

$ 198 , 000.00 

$ 478 , 167.65 

$ 1 , 434 , 502.95 

156 

3 

$ 94 , 439.14 

$ 283 , 317.41 

$ 198 , 000.00 

$ 481 , 317.41 

$ 1 , 443 , 952.23 

157 

3 

$ 107 , 161.53 

$ 321 , 484.58 

$ 198 , 000.00 

$ 519 , 484.58 

$ 1 , 558 , 453.74 

158 

3 

$ 77 , 829.81 

$ 233 , 489.42 

$ 198 , 000.00 

$ 431 , 489.42 

$ 1 , 294 , 468.26 

159 

3 

$ 103 , 747.47 

$ 311 , 242.40 

$ 198 , 000.00 

$ 509 , 242.40 

$ 1 , 527 , 727.19 

160 

3 

$ 120 , 507.79 

$ 361 , 523.36 

$ 198 , 000.00 

$ 559 , 523.36 

$ 1 , 678 , 570.08 

161 

3 

$ 112 , 889.85 

$ 338 , 669.54 

$ 198 , 000.00 

$ 536 , 669.54 

$ 1 , 610 , 008.61 

162 

3 

$ 89 , 608.94 

$ 268 , 826.83 

$ 198 , 000.00 

$ 466 , 826.83 

$ 1 , 400 , 480.48 

163 

3 

$ 106 , 088.45 

$ 318 , 265.35 

$ 198 , 000.00 

$ 516 , 265.35 

$ 1 , 548 , 796.06 

164 

3 

$ 97 , 556.00 

$ 292 , 668.01 

$ 198 , 000.00 

$ 490 , 668.01 

$ 1 , 472 , 004.04 

165 

3 

$ 94 , 841.53 

$ 284 , 524.60 

$ 198 , 000.00 

$ 482 , 524.60 

$ 1 , 447 , 573.79 

166 

3 

$ 112 , 021.79 

$ 336 , 065.37 

$ 198 , 000.00 

$ 534 , 065.37 

$ 1 , 602 , 196.10 

167 

3 

$ 110 , 524.60 

$ 331 , 573.81 

$ 198 , 000.00 

$ 529 , 573.81 

$ 1 , 588 , 721.42 

168 

3 

$ 107 , 175.48 

$ 321 , 526.45 

$ 198 , 000.00 

$ 519 , 526.45 

$ 1 , 558 , 579.35 

169 

3 

$ 93 , 742.31 

$ 281 , 226.93 

$ 198 , 000.00 

$ 479 , 226.93 

$ 1 , 437 , 680.78 

170 

3 

$ 83 , 454.82 

$ 250 , 364.45 

$ 198 , 000.00 

$ 448 , 364.45 

$ 1 , 345 , 093.34 

171 

3 

$ 105 , 832.20 

$ 317 , 496.61 

$ 198 , 000.00 

$ 515 , 496.61 

$ 1 , 546 , 489.82 

172 

3 

$ 106 , 947.90 

$ 320 , 843.70 

$ 198 , 000.00 

$ 518 , 843.70 

$ 1 , 556 , 531.10 

173 

3 

$ 100 , 777.76 

$ 302 , 333.29 

$ 198 , 000.00 

$ 500 , 333.29 

$ 1 , 500 , 999.86 

174 

3 

$ 98 , 847.69 

$ 296 , 543.06 

$ 198 , 000.00 

$ 494 , 543.06 

$ 1 , 483 , 629.18 

175 

3 

$ 100 , 389.74 

$ 301 , 169.23 

$ 198 , 000.00 

$ 499 , 169.23 

$ 1 , 497 , 507.69 

176 

3 

$ 109 , 309.65 

$ 327 , 928.96 

$ 198 , 000.00 

$ 525 , 928.96 

$ 1 , 577 , 786.87 

177 

3 

$ 81 , 658.93 

$ 244 , 976.79 

$ 198 , 000.00 

$ 442 , 976.79 

$ 1 , 328 , 930.36 

178 

3 

$ 96 , 848.77 

$ 290 , 546.30 

$ 198 , 000.00 

$ 488 , 546.30 

$ 1 , 465 , 638.89 

179 

3 

$ 66 , 776.04 

$ 200 , 328.12 

$ 198 , 000.00 

$ 398 , 328.12 

$ 1 , 194 , 984.35 

180 

3 

$ 75 , 776.95 

$ 227 , 330.85 

$ 198 , 000.00 

$ 425 , 330.85 

$ 1 , 275 , 992.54 

181 

3 

$ 80 , 311.45 

$ 240 , 934.35 

$ 198 , 000.00 

$ 438 , 934.35 

$ 1 , 316 , 803.05 

182 

3 

$ 78 , 479.62 

$ 235 , 438.86 

$ 198 , 000.00 

$ 433 , 438.86 

$ 1 , 300 , 316.59 

183 

3 

$ 64 , 660.47 

$ 193 , 981.40 

$ 198 , 000.00 

$ 391 , 981.40 

$ 1 , 175 , 944.19 

184 

3 

$ 82 , 974.17 

$ 248 , 922.50 

$ 198 , 000.00 

$ 446 , 922.50 

$ 1 , 340 , 767.49 

185 

3 

$ 54 , 419.76 

$ 163 , 259.28 

$ 198 , 000.00 

$ 361 , 259.28 

$ 1 , 083 , 777.83 

186 

3 

$ 67 , 788.25 

$ 203 , 364.76 

$ 198 , 000.00 

$ 401 , 364.76 

$ 1 , 204 , 094.28 

187 

3 

$ 72 , 785.63 

$ 218 , 356.89 

$ 198 , 000.00 

$ 416 , 356.89 

$ 1 , 249 , 070.66 

188 

3 

$ 78 , 140.21 

$ 234 , 420.62 

$ 198 , 000.00 

$ 432 , 420.62 

$ 1 , 297 , 261.86 

189 

3 

$ 77 , 090.29 

$ 231 , 270.86 

$ 198 , 000.00 

$ 429 , 270.86 

$ 1 , 287 , 812.59 

190 

3 

$ 64 , 409.49 

$ 193 , 228.48 

$ 198 , 000.00 

$ 391 , 228.48 

$ 1 , 173 , 685.43 

191 

3 

$ 93 , 720.78 

$ 281 , 162.34 

$ 198 , 000.00 

$ 479 , 162.34 

$ 1 , 437 , 487.02 

192 

3 

$ 67 , 783.18 

$ 203 , 349.54 

$ 198 , 000.00 

$ 401 , 349.54 

$ 1 , 204 , 048.61 

193 

3 

$ 51 , 021.64 

$ 153 , 064.91 

$ 198 , 000.00 

$ 351 , 064.91 

$ 1 , 053 , 194.73 
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194 

3 

$ 58 , 644.91 

$ 175 , 934.73 

$ 198 , 000.00 

$ 373 , 934.73 

$ 1 , 121 , 804.19 

195 

4 

$ 65 , 440.97 

$ 196 , 322.92 

$ 198 , 000.00 

$ 394 , 322.92 

$ 1 , 577 , 291.67 

196 

4 

$ 73 , 954.14 

$ 221 , 862.43 

$ 198 , 000.00 

$ 419 , 862.43 

$ 1 , 679 , 449.72 

197 

4 

$ 76 , 687.89 

$ 230 , 063.67 

$ 198 , 000.00 

$ 428 , 063.67 

$ 1 , 712 , 254.69 

198 

4 

$ 59 , 507.63 

$ 178 , 522.90 

$ 198 , 000.00 

$ 376 , 522.90 

$ 1 , 506 , 091.61 

199 

4 

$ 61 , 010.15 

$ 183 , 030.46 

$ 198 , 000.00 

$ 381 , 030.46 

$ 1 , 524 , 121.85 

200 

4 

$ 64 , 353.94 

$ 193 , 061.82 

$ 198 , 000.00 

$ 391 , 061.82 

$ 1 , 564 , 247.29 

201 

4 

$ 77 , 788.33 

$ 233 , 365.00 

$ 198 , 000.00 

$ 431 , 365.00 

$ 1 , 725 , 460.02 

202 

4 

$ 88 , 095.77 

$ 264 , 287.31 

$ 198 , 000.00 

$ 462 , 287.31 

$ 1 , 849 , 149.25 

203 

4 

$ 65 , 718.38 

$ 197 , 155.15 

$ 198 , 000.00 

$ 395 , 155.15 

$ 1 , 580 , 620.61 

204 

4 

$ 64 , 767.91 

$ 194 , 303.73 

$ 198 , 000.00 

$ 392 , 303.73 

$ 1 , 569 , 214.92 

205 

4 

$ 70 , 751.66 

$ 212 , 254.99 

$ 198 , 000.00 

$ 410 , 254.99 

$ 1 , 641 , 019.94 

206 

4 

$ 72 , 728.67 

$ 218 , 186.00 

$ 198 , 000.00 

$ 416 , 186.00 

$ 1 , 664 , 743.99 

207 

4 

$ 71 , 139.68 

$ 213 , 419.04 

$ 198 , 000.00 

$ 411 , 419.04 

$ 1 , 645 , 676.17 

208 

4 

$ 62 , 219.77 

$ 186 , 659.31 

$ 198 , 000.00 

$ 384 , 659.31 

$ 1 , 538 , 637.25 

209 

4 

$ 89 , 870.50 

$ 269 , 611.49 

$ 198 , 000.00 

$ 467 , 611.49 

$ 1 , 870 , 445.94 

210 

4 

$ 74 , 680.66 

$ 224 , 041.97 

$ 198 , 000.00 

$ 422 , 041.97 

$ 1 , 688 , 167.90 

211 

4 

$ 104 , 754.60 

$ 314 , 263.81 

$ 198 , 000.00 

$ 512 , 263.81 

$ 2 , 049 , 055.26 

212 

4 

$ 95 , 773.64 

$ 287 , 320.91 

$ 198 , 000.00 

$ 485 , 320.91 

$ 1 , 941 , 283.65 

213 

4 

$ 91 , 223.31 

$ 273 , 669.92 

$ 198 , 000.00 

$ 471 , 669.92 

$ 1 , 886 , 679.69 

214 

4 

$ 93 , 049.80 

$ 279 , 149.41 

$ 198 , 000.00 

$ 477 , 149.41 

$ 1 , 908 , 597.62 

215 

4 

$ 106 , 868.96 

$ 320 , 606.87 

$ 198 , 000.00 

$ 518 , 606.87 

$ 2 , 074 , 427.50 

216 

4 

$ 88 , 555.26 

$ 265 , 665.77 

$ 198 , 000.00 

$ 463 , 665.77 

$ 1 , 854 , 663.09 

217 

4 

$ 117 , 294.83 

$ 351 , 884.49 

$ 198 , 000.00 

$ 549 , 884.49 

$ 2 , 199 , 537.97 

218 

4 

$ 103 , 741.17 

$ 311 , 223.51 

$ 198 , 000.00 

$ 509 , 223.51 

$ 2 , 036 , 894.05 

219 

4 

$ 98 , 743.79 

$ 296 , 231.38 

$ 198 , 000.00 

$ 494 , 231.38 

$ 1 , 976 , 925.53 

220 

4 

$ 93 , 389.22 

$ 280 , 167.65 

$ 198 , 000.00 

$ 478 , 167.65 

$ 1 , 912 , 670.60 

221 

4 

$ 94 , 439.14 

$ 283 , 317.41 

$ 198 , 000.00 

$ 481 , 317.41 

$ 1 , 925 , 269.63 

222 

4 

$ 107 , 161.53 

$ 321 , 484.58 

$ 198 , 000.00 

$ 519 , 484.58 

$ 2 , 077 , 938.32 

223 

4 

$ 77 , 829.81 

$ 233 , 489.42 

$ 198 , 000.00 

$ 431 , 489.42 

$ 1 , 725 , 957.68 

224 

4 

$ 103 , 747.47 

$ 311 , 242.40 

$ 198 , 000.00 

$ 509 , 242.40 

$ 2 , 036 , 969.58 

225 

4 

$ 120 , 507.79 

$ 361 , 523.36 

$ 198 , 000.00 

$ 559 , 523.36 

$ 2 , 238 , 093.44 

226 

4 

$ 112 , 889.85 

$ 338 , 669.54 

$ 198 , 000.00 

$ 536 , 669.54 

$ 2 , 146 , 678.15 

227 

4 

$ 89 , 608.94 

$ 268 , 826.83 

$ 198 , 000.00 

$ 466 , 826.83 

$ 1 , 867 , 307.31 

228 

4 

$ 106 , 088.45 

$ 318 , 265.35 

$ 198 , 000.00 

$ 516 , 265.35 

$ 2 , 065 , 061.41 

229 

4 

$ 97 , 556.00 

$ 292 , 668.01 

$ 198 , 000.00 

$ 490 , 668.01 

$ 1 , 962 , 672.06 

230 

4 

$ 94 , 841.53 

$ 284 , 524.60 

$ 198 , 000.00 

$ 482 , 524.60 

$ 1 , 930 , 098.39 

231 

4 

$ 112 , 021.79 

$ 336 , 065.37 

$ 198 , 000.00 

$ 534 , 065.37 

$ 2 , 136 , 261.47 

232 

4 

$ 110 , 524.60 

$ 331 , 573.81 

$ 198 , 000.00 

$ 529 , 573.81 

$ 2 , 118 , 295.23 

233 

4 

$ 107 , 175.48 

$ 321 , 526.45 

$ 198 , 000.00 

$ 519 , 526.45 

$ 2 , 078 , 105.80 
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234 

4 

$ 93 , 742.31 

$ 281 , 226.93 

$ 198 , 000.00 

$ 479 , 226.93 

$ 1 , 916 , 907.71 

235 

4 

$ 83 , 454.82 

$ 250 , 364.45 

$ 198 , 000.00 

$ 448 , 364.45 

$ 1 , 793 , 457.79 

236 

4 

$ 105 , 832.20 

$ 317 , 496.61 

$ 198 , 000.00 

$ 515 , 496.61 

$ 2 , 061 , 986.43 

237 

4 

$ 106 , 947.90 

$ 320 , 843.70 

$ 198 , 000.00 

$ 518 , 843.70 

$ 2 , 075 , 374.80 

238 

4 

$ 100 , 777.76 

$ 302 , 333.29 

$ 198 , 000.00 

$ 500 , 333.29 

$ 2 , 001 , 333.14 

239 

4 

$ 98 , 847.69 

$ 296 , 543.06 

$ 198 , 000.00 

$ 494 , 543.06 

$ 1 , 978 , 172.23 

240 

4 

$ 100 , 389.74 

$ 301 , 169.23 

$ 198 , 000.00 

$ 499 , 169.23 

$ 1 , 996 , 676.91 

241 

4 

$ 109 , 309.65 

$ 327 , 928.96 

$ 198 , 000.00 

$ 525 , 928.96 

$ 2 , 103 , 715.83 

242 

4 

$ 81 , 658.93 

$ 244 , 976.79 

$ 198 , 000.00 

$ 442 , 976.79 

$ 1 , 771 , 907.14 

243 

4 

$ 96 , 848.77 

$ 290 , 546.30 

$ 198 , 000.00 

$ 488 , 546.30 

$ 1 , 954 , 185.19 

244 

4 

$ 66 , 776.04 

$ 200 , 328.12 

$ 198 , 000.00 

$ 398 , 328.12 

$ 1 , 593 , 312.47 

245 

4 

$ 75 , 776.95 

$ 227 , 330.85 

$ 198 , 000.00 

$ 425 , 330.85 

$ 1 , 701 , 323.39 

246 

4 

$ 80 , 311.45 

$ 240 , 934.35 

$ 198 , 000.00 

$ 438 , 934.35 

$ 1 , 755 , 737.39 

247 

4 

$ 78 , 479.62 

$ 235 , 438.86 

$ 198 , 000.00 

$ 433 , 438.86 

$ 1 , 733 , 755.46 

248 

4 

$ 64 , 660.47 

$ 193 , 981.40 

$ 198 , 000.00 

$ 391 , 981.40 

$ 1 , 567 , 925.58 

249 

4 

$ 82 , 974.17 

$ 248 , 922.50 

$ 198 , 000.00 

$ 446 , 922.50 

$ 1 , 787 , 689.99 

250 

4 

$ 54 , 419.76 

$ 163 , 259.28 

$ 198 , 000.00 

$ 361 , 259.28 

$ 1 , 445 , 037.11 

251 

4 

$ 67 , 788.25 

$ 203 , 364.76 

$ 198 , 000.00 

$ 401 , 364.76 

$ 1 , 605 , 459.04 

252 

4 

$ 72 , 785.63 

$ 218 , 356.89 

$ 198 , 000.00 

$ 416 , 356.89 

$ 1 , 665 , 427.55 

253 

4 

$ 78 , 140.21 

$ 234 , 420.62 

$ 198 , 000.00 

$ 432 , 420.62 

$ 1 , 729 , 682.48 

254 

4 

$ 77 , 090.29 

$ 231 , 270.86 

$ 198 , 000.00 

$ 429 , 270.86 

$ 1 , 717 , 083.45 

255 

4 

$ 64 , 409.49 

$ 193 , 228.48 

$ 198 , 000.00 

$ 391 , 228.48 

$ 1 , 564 , 913.91 

256 

4 

$ 93 , 720.78 

$ 281 , 162.34 

$ 198 , 000.00 

$ 479 , 162.34 

$ 1 , 916 , 649.36 

257 

4 

$ 67 , 783.18 

$ 203 , 349.54 

$ 198 , 000.00 

$ 401 , 349.54 

$ 1 , 605 , 398.14 

258 

4 

$ 51 , 021.64 

$ 153 , 064.91 

$ 198 , 000.00 

$ 351 , 064.91 

$ 1 , 404 , 259.65 

259 

4 

$ 58 , 644.91 

$ 175 , 934.73 

$ 198 , 000.00 

$ 373 , 934.73 

$ 1 , 495 , 738.93 


See text for data collection methods and details. 
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APPENDIX G. PROBABILITY OF MISSION SUCCESS 



DP# 

Success 

P hat 

Lower Cl 

95% 

Relative Cost 

0 

79 

99 

0.99 

0.97040 

$935,222.97 

1 

106 

77 

0.77 

0.68710 

$1,030,993.22 

2 

110 

89 

0.89 

0.82836 

$998,338.46 

3 

140 

81 

0.81 

0.73272 

$1,230,764.96 

4 

141 

59 

0.59 

0.49312 

$1,248,557.99 

5 

144 

93 

0.93 

0.87974 

$1,402,834.46 

6 

150 

64 

0.64 

0.54545 

$1,555,820.62 

7 

159 

60 

0.6 

0.50350 

$1,527,727.19 

8 

163 

78 

0.78 

0.69840 

$1,548,796.06 

9 

171 

88 

0.88 

0.81599 

$1,546,489.82 

10 

172 

60 

0.6 

0.50350 

$1,556,531.10 

11 

175 

92 

0.92 

0.86656 

$1,497,507.69 

12 

176 

54 

0.54 

0.44182 

$1,577,786.87 

13 

201 

99 

0.99 

0.97040 

$1,725,460.02 

14 

205 

100 

1 

1.00000 

$1,641,019.94 

15 

206 

100 

1 

1.00000 

$1,664,743.99 

16 

209 

100 

1 

1.00000 

$1,870,445.94 

17 

211 

97 

0.97 

0.93640 

$2,049,055.26 

18 

212 

83 

0.83 

0.75601 

$1,941,283.65 

19 

215 

100 

1 

1.00000 

$2,074,427.50 

20 

221 

52 

0.52 

0.42159 

$1,925,269.63 

21 

224 

100 

1 

1.00000 

$2,036,969.58 

22 

226 

71 

0.71 

0.62061 

$2,146,678.15 

23 

227 

88 

0.88 

0.81599 

$1,867,307.31 

24 

228 

100 

1 

1.00000 

$2,065,061.41 

25 

229 

81 

0.81 

0.73272 

$1,962,672.06 

26 

233 

96 

0.96 

0.92140 

$2,078,105.80 

27 

236 

100 

1 

1.00000 

$2,061,986.43 

28 

237 

99 

0.99 

0.97040 

$2,075,374.80 

29 

240 

100 

1 

1.00000 

$1,996,676.91 

30 

241 

100 

1 

1.00000 

$2,103,715.83 

31 

246 

77 

0.77 

0.68710 

$1,755,737.39 


See text for data collection methods and details. 
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